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Forward 



About JALT Applied Materials 

Welcome to the JALT Applied Materials (JAM) series, of which Classroom Teachers 
and Classroom Research is the second volume. The concept beliind JAM is that 
each proposed volume is a collection of papers focused on a single theme, written 
by classroom teachers, some experienced and some just beginning. 



The Situation 

Why begin a new series? At present, books and collections of papers on various 
subjects are written and published by authors from various parts of the world, 
mainly from English-speaking countries such as Australia, Canada, New Zealand, 
the U.K. and the U.S. The authors of these books and articles are typically 
university professors who teach courses in graduate programs and speak at 
conferences around the world including JALT’s annual international conference. 
There exists, however, a gap between the international level where these books 
are generated and written and the local teaching community in Japan and Asia. 
While these books are of general professional interest, local teachers are usually 
not sought as contributors, and only find out about the book when they see it in 
a book store or catalog. The JAM series was created too help fill this gap. 



Purpose 

The JAM series is targeted at: (a) improving the quality of research and aca- 
demic writing in Japan and Asia; (b) publishing collections of theoretically 
grounded, reader- friendly, and classroom oriented articles on subjects of inter- 
est to classroom teachers in Japan and Asia; (c) giving Japan and Asia based 
classroom teachers a publication outlet not heretofore available; and (d) help- 
ing teachers around the world implement new ideas in their classrooms. 
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How JAM Works 

We feel that JALT has reached a level of development where many of its mem- 
bers are capable of writing creative and professional papers. Research and writ- 
ing, however, are not easy tasks, and we feel that JALT members would be 
helped to produce quality papers if they had comprehensive and constaictive 
editorial support and guidance. To this end, the JAM series is guided by an 
editorial network composed of a general series editor, one or more local edi- 
tors, and the possibility of an international editor for each collection of papers, 
all of whom would work with the writers of that collection. 

Any person who is a resident of Japan, a member of JALT, and primarily a 
classroom teacher may request to be a JAM editor. Persons requesting to be 
editors would be asked to submit their CV and publications relevant to their 
subject. This Japan-based local editor (or editors) could, if they desired, ask an 
international expert in the field to assist them. We will refer to this editor as the 
“global” editor. After approval by the JAM committee of the theme and editors, 
the next step would be for the local editor to issue a call for abstracts. The date 
for first drafts would be set by the local editor and the series editor. All manu- 
scripts must be approved by all editors: first by the local editor, then by the 
global editor, and finally by the series editor. Thus, JAM is a vetted publication, 
and it will be a personal as well as professional achievement to have an article 
published in a JAM collection. 

Contributing authors to a JAM collection are expected to be classroom teach- 
ers in Japan or a country with similar teaching conditions (e.g., Korea, China, 
Taiwan, or Thailand), to be familiar with JALT and its mission, and to submit an 
original article which has not been submitted to another publication. JAM wel- 
comes new as well as experienced writers, and JAM authors need not have 
published previously in the theme area. 



Conclusion:A Unique Opportunity 

JALT is a professional organization of teachers who occupy the niche between 
new developments and traditional classroom practices. As such, we are histori- 
cally positioned to creatively interact between theory and the daily reality of the 
classroom. We firmly believe that what we have to report will be of significant 
value to teachers throughout Asia and around the world. We are confident that 
you will concur. 

N.B. Persons who desire further information about JAM should contact the se- 
ries editor. Dale. T Griffee at Seigakuin University, 1-1 Tosaki, Ageo-shi, Saitama- 
ken 362, Japan. 
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SECTION I 



An Overview 





Chapter 1 



Language Teaching and Research 



David Nunan 

University of Hong Kong 



In this introductory paper, I should like to do three things. In the first place, I 
should like to articulate a vision of research which is in harmony with second 
and foreign language education. Secondly, I should like to summarize what I 
see as some of the things we have learned from research that can help us as we 
staiggle to improve what we do in the classroom. The final thing I should like 
to do is to set out what I see as some important future trends for applied 
research in language education. 



Approaches to Research 

It is something of a curiosity to me that few of the recent spate of books on 
research in language education attempt to offer a definition of the term itself. 
So, years ago I asked a group of my graduate students, all language teachers 
who were embarking on an M.A. in Applied Linguistics, what the term meant 
for them. Some of their responses are set out below. 

• Enquiry has two components; process and product. The process is about an 
area of enquiry and the process used to pursue that. The product is the knowl- 
edge generated from the process as well as the initial area to be presented. 

• A process which involves (a) defining a problem, (b) stating an objective, 
and (c) formulating an hypothesis. It involves gathering information, classi- 
fication, analysis, and interpretation to see to what extent the initial objec- 
tive has been achieved. 
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• Undertaking structured investigation which hopefully results in greater un- 
derstanding of the chosen interest area. Ultimately, this investigation be- 
comes accessible to the public. 

• An activity which analyses and critically evaluates some problem. 

• To collect and analyse the data in a specific field with the purpose of prov- 
ing your theory. 

• Evaluation, asking questions, investigations, analysis, confirming hypotheses, 
oveiview, gathering and analysing data in a specific field according to cer- 
tain predetermined methods. 

For me, research is a systematic process of inquiry in which the researcher 
poses a question or questions, collects relevant data on the question(s), analy- 
ses and interprets the questions, and makes the results of the inquiry publicly 
accessible in some way. 

Donald Freeman, who has written extensively on research in TESOL is one 
person who has offered a characterization of research suggesting that it is “a 
basic process of developing and rendering viable interpretations for things in 
the world” (Freeman, 1996, p. 102). While this is a normal and natural part of 
everyday life, research differs from normal human curiosity through its 
systematicity and accessibility to scmtiny. In practical terms, research is a pro- 
cess of formulating questions or articulating “puzzles” relating to practice, col- 
lecting relevant data that might have a bearing on such questions or puzzles, 
interpreting and explaining the data, and making the results of the inquiry 
publicly accessible in some way (Nunan, 1992a). As Freeman pointed out, to 
give this process the label “research” is important because it gives the activity 
value. “When [the] questioning of practice takes place within a framework la- 
beled ‘research,’ understanding the complexity of teaching can become a public 
and legitimate part of being a teacher” (Freeman, 1996, p. 103). 

Many commentators argue that there are two competing traditions in re- 
search. The first of these, the quantitative tradition, is obtmsive and controlled 
and is concerned with established generalizable relationships between vari- 
ables. The other, the qualitative tradition, is concerned with generating insight 
and understanding rather than establishing “truths.” More recently, it has been 
argued that this distinction is simplistic and naive. Elsewhere, I have suggested 
that while the distinction was simplistic it is also persistent: 

One reason for the persistence of the distinction between quantitative and qualita- 
tive research, is that the two approaches represent different ways of thinking about 
and understanding the world around us. Underlying the development of different 
research traditions and methods is a debate on the nature of knowledge, and the 
status of assertions about the world, and the debate itself is ultimately a philo- 
sophical one. It is commonly assumed that the function of research is to add to our 
knowledge of the world and to demonstrate the ‘truth’ of the commonsense no- 
tions we have about the world. ... In developing one’s own philosophy on 
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research, it is iniportant for us to determine where we stand on the notion of ‘truth’ 
in relation to research. What is taith? (Even more basically, do we accept that there 
is such a thing as ‘taith’?) What is evidence? Can we ever ‘prove’ anything? What 
evidence would compel us to accept the truth of an assertion or proposition? 
These are questions which need to be borne constantly in mind as one reads and 
evaluates research. (Nunan, 1992a, p. 10) 



What Have We Learned From Research? 

In conducting teacher education seminars on research methods, I am sometimes 
confronted with the view that research is an esoteric activity that has little to do 
with the real world, and little to say to the classroom practitioner. I beg to differ. 
While honest research is messy, the results often inconsistent if not contradictory, 
and the outcomes uncertain, research has helped to advance our understanding of 
the processes underlying language acquisition. I also believe that research can 
offer us an empirical basis for piactice. At a recent conference in Brazil (Nunan, 
1996b), I was asked to address the relationship of research to practice, and to spell 
out the practical implications of research for language pedagogy. I gave my own 
idiosyncratic list of principles that guide my teacliing and materials writing that are 
derived from my own research and that of others. These I have reproduced below, 
along with the research from wliich they are derived. 

1. Make instructional goals explicit to learners in ways they can understand. 

• Goal setting can have exceptional importance in stimulating L2 learning and 
enhancing motivation (Oxford & Shearin, 1993). 

• Motivation is enhanced when learning goals are made clear (Reilly, 1994). 

• Learning was enhanced in classes where teachers made goals explicit. Ex- 
plicit goal setting was relatively rare in classrooms observed (Nunan, 1996a). 

2. Give learners opportunities to make their own contributions to the learning 
process. 

• Learners have definite views on what they want to learn and how they want 
to learn. These are often at variance with the views of the teacher. Learning 
is enhanced when learners are involved in making choices (Nunan, 1987). 

• Young learners are able to take responsibility for planning, organising, man- 
aging and evaluating their own learning (Dam & Gabrielsen, 1988). 

• Students want to be involved in the selection of language content and the 
learning process. There are major mismatches between the desires of the 
students and the mandated curriculum (Widdows & Voller, 1991). 

• Classroom topics nominated by learners were much more likely to be learned 
than topics determined by the teacher (Slimani, 1992). 

• Learning is enhanced when students are actively involved in selecting con- 
tent, learning tasks, and evaluation (Heath, 1992). 




14 



16 



David Nunan 



3 . Encourage active communication through sequenced, achievable tasks. 

• Spoken output in groupwork is a significant factor to the acquisition of 
vocabulary (Hall, 1991). 

• Learner participation in class relates significantly to improvements in lan- 
guage proficiency (Lim, 1992). 

• Motivation is enhanced when tasks are sequenced and linked in ways that 
make sense to the learner (Reilly, 1994). 

• Active use of target language with strong emphasis on practice in naturalis- 
tic situations is notable in helping students attain higher proficiency (Green 
& Oxford, 1995). 

4. Provide opportunities for learners to apply their skills beyond the classroom. 

• Supplementing classroom instruction with out-of-class tasks results in sig- 
nificantly increased language gains (Montgomery & Eisenstein, 1985). 

• InstRiction and opportunities to communicate out of class are both neces- 
sary. Improvement occurred when subject consciously “noticed the gap” 
(Schmidt & Frota, 1986). 

• The “good” foreign language learner finds ways of activating his or her 
language out of class (Nunan, 1991). 

5. Teach learning strategies as well as language content. 

• The ability to infer or induct rules is an important aspect of language apti- 
tude (Carroll, 1981). 

• Strategy training had a significant effect on speaking development (O’Malley, 
Chamot, Stewner-Manzanares, Russo, & Kupper, 1985). 

• Effective learners are aware of the processes underlying their own learning 
and seek to use appropriate learning strategies to control their own learn- 
ing Qones, Palincsar, Ogle, & Carr, 1987). 

• More effective learners use strategies more frequently, and use a greater 
variety of strategies than students designated as less effective learners 
(O’Malley & Chamot, 1990). 

• Good learners are aware of strategies that worked for them, and can articu- 
late these (Nunan, 1991). 

• There is a significant relationship between strategy use and success in lan- 
guage learning (Green & Oxford, 1995). 

6. Teach grammar in ways that show the essential harmony between form 

and function. 

Grammar exists in order to enable learners to communicate in increasingly 
sophisticated ways (Nunan, 1993). 

Grammar and vocabulary are best acquired through learner involvement in 
the processing and production of discourse (McCarthy & Carter, 1994). 
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7. Go heyoi'id declarative knoiuledge to procedural skills development. 

• "Communicative” classrooms with instruction plus opportunities for inter- 
action are superior to “traditional” instaiction and also to immersion pro- 
grams (Spacla, 1990). 

• Formal instruction resulted in acquisition of some structures (passives) but 
not others (tense and aspect). Explicit (declarative) knowledge can be con- 
verted to implicit (procedural) knowledge through practice (Zhou, 1991). 

• Declarative knowledge (ability to identify errors and state aile violation) 
does not lead to procedural knowledge (ability to put known forms to com- 
municative effect) without opportunities to activate knowledge through out- 
put activities (Wudong, 1994). 

8. Give learne?^ an opportunity to work with authentic data. 

• Non-authentic data misrepresents the nature of genuine communication 
(Nunan, 1991)- 

• Authenticity significantly enhances motivation in foreign language classes 
(Ho, 1995). 

9. Maximize opportunities for learners to work cooperatively. 

• Small group tasks prompt students to use a greater range of language func- 
tions than teacher fronted tasks (Long, Adams, & Castanos, 1976). 

• Cooperative learning leads to positive interdependence but also individual 
accountability, extensive face-to-face interaction, and the development of 
social skills (Kohonen, 1992). 

:10. Provide learners with opportunities to self -monitor and self -check. 

• Opportunities for learners to self-monitor and self-check lead to greater sen- 
sitivity to the learning process. Learners develop skills in articulating what 
they want to learn and how they want to learn (Nunan, 1995). 



Future Trends 

And so to the future. What direction would I like to see research in language 
education taking? I believe tliat a future research agenda should be contextualised 
and classroom oriented. It should forge closer links between teaching and research 
of the type indicated in the body of tliis paper. It should encourage collaboration 
iDetween researchers, teachers and students. And, fmally, it should seek to extend 
our vision of research. Each of tliese points is commented on below. 

1. Contextualized and classroom-oriented. 

The need for research to be contextualized is slowly being recognised. Even 
university-based researchers with little direct contact to classroom realities are 
beginning to recognise the limited applicability of research outcomes which 
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have been derived from contexts other than those in which teaching and learn- 
ing typically occur. To acknowledge that context affects behavior is to acknowl- 
edge that conclusions reached in one context can be taken as nothing more 
than working hypotheses in other contexts. 

2. Closer links hetiveen teaching and research. 

Related to the first point is an appreciation of the need for developing closer 
links between teaching and research. Allwright and Bailey (1991) coined the 
phrase “exploratory^ teaching” to capture this alternative vision in which it is 
unnecessaiy to decide whether one is engaged in teaching or research; rather 
one is engaged in both. In other words, exploratory teaching describes a philo- 
sophical stance or attitude of mind towards one’s classroom practice. It is an 
ongoing process of working constantly for deeper understanding and increas- 
ing effectiveness in the classroom. Underlying tlieir approach to reflective teaching 
are the assumptions that (a) autonomous teachers are the key link between 
teaching, learning, and research, and (b) becoming more effective as a teacher 
is a life-long, spiraling process. 

The teacher is the researcher’s link with learners, and also the learners’ link with 
research. The teacher is contracted to help learners learn, but can do so better by 
knowing about previous research and by using the procedures of classroom re- 
search to understand better what is happening in his or her own classroom. In this 
way the exploratory teacher will not only improve achievement but will also con- 
tribute to our general research knowledge about how language classrooms work. 
This is what we mean by ‘exploratory teaching’ — teaching that not only tries out 
new ideas, but that also tries to learn as much as possible from doing so. . . . Any 
good experienced teacher will no doubt spend a lot of class time on ideas that are 
tried and trusted. Turning that ‘good’ teaching into ‘exploratory’ teaching is a mat- 
ter of tiying to find out what makes the tried and taisted ideas successful. (Allwright 
<5:^ Bailey, 1991, p. 197) 

3- Collaborative research. 

Collaborative research is research in which all those involved in the research 
process, teacher-researchers, administrators, and informants have an active role 
and a voice within the process itself. The term also refers to inquiry carried out 
by multiple participants across a number of teaching sites. The value of col- 
laboration is slowly being recognised, and is something which the field should 
seek to encourage. Such collaboration can be facilitated by the networks and 
systems (electronic and othei*wise), which can help to link together individuals 
who might be working in different contexts (and even different countries). 
Teacher-researchers who collaborate can generate more powerful ideas, ini- 
tiate more interesting research, and achieve more useful outcomes, than those 
working in isolation. A useful question for consideration is: “What are the 
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central characteristics of a collaborative approach to classroom research, and 
in what ways do the different contributions of teachers, learners and research- 
ers provide us with insights which would be difficult to obtain in any other 
way?” (Nunan, 1992b, p. 8). 

4. An extended vision of the nature of research. 

There is growing acceptance within the social sciences generally that research- 
ers need to develop a greater range of models, tools, and analytical techniques. 
Teacher-researchers working in language education should develop an extended 
vision of the nature of research. They should extend the theoretical bases of the 
research as well as the range of research tools, techniques, and methods. While 
conventional research methods — experiments, etlinographies, case studies and 
the like — have their place within a practitioner-oriented research agenda, there 
is a need to go beyond these. Ways of extending the agenda exist in the litera- 
ture. These include focused teaching (Allwright & Bailey, 1991), action research 
(Nunan, 1989), narrative accounts (Freeman, 1994). 

The four key principles set out in this section underlie my vision of the kind 
of research agenda I would like to develop and promote. Such an agenda, 
while not rejecting quantitative research, seeks to enhance the status of 
constructivist, humanistic research paradigms which are focused on understanding 
and explaining the contexts in which language education is conducted around 
the world. Insight and understanding, rather than causality and proof capture 
the essence of the perspective I am trying to promote. 
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Trends, Teachers, and 
Classroom Research 
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Imagine that you have been hired by JR O^P^n Rail) to teach a special class on 
one of their newest express trains. You teach in a special classroom which has 
been built in the train. You glance out the window as the train pulls out of the 
station and then turn your attention to your class. The class goes well and the 
hours rty by. You take a break and look out the window, but the scenery is 
different and you wonder to yourself, where are we now? 

As the Japan Association for Language Teaching QALT) faces the 21st cen- 
tuiy and the beginning of the JALT Applied Materials 0AM) series, it seems 
appropriate to ask the question, where are we now and by implication, where 
are we going? The working hypothesis for this article is that being a language 
teacher in Japan is like riding a fast moving train. Every so often it is a good 
idea to look out the window and reflect on just where we are, where we are 
going, and perhaps what we might do when we arrive. This article will deal 
with three questions. The first question is, what trends are being noticed and 
articulated by the teaching profession at large?; the second question is, to what 
extent does the JALT leadership agree or disagree with these trends?; and the 
third question is, what additional trends do JALT leaders perceive? 
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Definitions 

By the word “trend,” I mean an issue that is identified as a trend by one or 
more sources in the literature. By the term “teaching profession at large,” I mean 
those persons teaching English as a second language anywhere in the world 
including Japan, especially those persons who are members of teaching organi- 
zations such as TESOL, lATEFL, and JALT, and who write articles that appear in 
the journals such as those published by these organizations. By the term “we” 
as in tlie sentence, “Where are we now?” I include the teaching profession at 
large with special emphasis on JALT members. For the purpose of this article, 
the term “JALT leadership,” will be taken to mean those persons listed as chap- 
ter officers, SIG officers, elected national officers, and key committee persons in 
the April, 1994, supplement to The Language Teacher. Tliis is not an altogether 
inclusive definition of JALT leadership but it has the advantage of being a rela- 
tively small group, it is a group whose names and addresses are publicly listed, 
and because of the offices these individuals hold they can reasonably be ex- 
pected to be interested in the issue of professional trends. 

The Trends 

My first question is, what trends are being noticed and attended to by the 
teaching profession at large? From the literature, 15 trends were identified. 

1. As a knowledge-based industry, we are in the process of becoming part of 
the global information network (Ashworth, 1991; Bowers, 1994; Fanselow, 
1987; Schinke-Llano, 1991; Widdowson, 1986, 1992). 

2. We are experiencing dramatic growth in numbers in our field (Alatis, 1987; 
Ashworth, 1991; Schinke-Llano, 1991; Swales, 1993). 

3. The new Monbusho (Ministry of Education, Science and Welfare) course 
guidelines will have a significant impact (Aiga, 1990; Carter, Goold, & 
Madeley, 1993; Goold, Madeley, & Carter, 1993; Wada, 1994, 1995; Wada & 
Cominos, 1994; Wadden, 1994). 

4. The emergence of U. S. colleges and universities in Japan (Helgesen, 1991). 

5. The JET, AET program will have an influence (Aiga, 1990; Cominos, 1992; 
Leedham, 1991; Shang-Ikeda, 1992; Wada, 1994,1995; Wada & Cominos, 
1994; Yukawa, 1992). 

6. More teachers are becoming interested in classroom based research 
(Allwright, 1994; Gaies, 1991; Genesee, 1994; Grosse, 1991; McDonough & 
McDonough, 1990; Nunan, 1989). 

7. The relationship between theory and practice is becoming closer (Clarke, 
1994; Ellis, 1992; Gaies, 1987; McDonough & McDonough, 1990; Schinke- 
Llano, 1991; Wagner, 1991; Ur, 1992). 

8. There is a feeling of being adrift between two worlds because we don’t 
have a theory to make sense of or to justify our teaching (O’Neill, 1989). 

9. There is a trend towards a focus on the learner as an individual. For ex- 
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ample, learner training and learner development (Brown, 1991; Morley, 
1987; Savignon, 1991). 

10. There is a trend toward seeing teaching as unique and as having its own 
skills and knowledge (Freeman Richards, 1993; Gaies, 1991; Genesee, 
1994). 

11. There is an increasing awareness that we are a separate discipline and not a 
branch of linguistics (Bahns, 1990; Grosse, 1991; Nunan, 1989; Richards <& 
Rogers, 1987; Schinke-Llano, 1991; Ur, 1992). 

12. We are becoming increasingly professional and more concerned with issues 
such as the development of credentials, regulations, and entry into the pro- 
fession (Aitchison, 1993; Ashworth, 1991; Bowers, 1994; Clayton, 1989; Gaies, 
1987; Grosse, 1991; Helgesen, 1991; Maley, 1992; O’Neill, 1990; Redfield, 
1990; Swales, 1993). 

13. There is a trend towards nonnative speakers playing a more important role 
in the teaching of English, Already most EFL teachers in the world are non- 
native speakers (Brown, 1991). 

14. There is a trend towards including human issues in our curriculum such as 
peace and environmental education (Brown, 1991). 

15. Psycholinguistics is developing into a more important area for language teach- 
ers (Aitchison, 1993). 

This list of trends, while well documented in the literature, is not a complete list 

and perhaps could never be. Rather, it represents one possible set of answers to 

the questions, where are we now and where are we heading? 



The Response to the Suwey 

My second question is, to what extent do JALT leaders agree or disagree with 
these trends. Eighty questionnaires were mailed to persons listed as National 
Officers, N-SIG Officers, and Chapter Officers. As Table 1 shows, surveys were 
sent to all the National Officers, one representative from each SIG and one 
representative from each Chapter. The survey was mailed to the person in the 
SIG or the Chapter who was listed as president or coordinator unless that per- 
son was also listed as a National Officers in which case the survey was sent to 
the program chair. Fifty-five sui*vey questionnaires were returned for a response 
rate of sixty-nine percent (see Table 1). 

Of the 55 valid respondents, the LI was given as English by 47, Japanese by 
6, French by 1, and German by 1. There were 39 male respondents and l6 
females. The average years teaching experience was 14.5 years. Some of the 
respondents teach at more than one type of institution. Five teach at junior high 
schools, ten at high schools, thirty-four at colleges or universities, eight at com- 
mercial language schools, five at home, four at senmongakko (vocational schools), 
and seven teach in business in-house programs. The most typical respondent 
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Table 1 

Sui*vey response 





mailed 


returned 

invalid 


returned 


per cent 
valid 


National Officers 


8 




8 


1.0 


Appointed Officers 


10 


1 


8 


.80 


Committee Chairs 


7 




6 


.86 


Editors 


5 




3 


.60 


JALT 94 Conference 


4 


1 


3 


.75 


N-SIG Coordinators 


10 




6 


.60 


Chapter Officers 


36 


1 


21 


.58 


Totals 


80 


3 


55 


.69 



Nolc: One survey was relumed due lo an incorrect address and two surveys were returned unanswered. 



was an English speaking male working at a college or a university who has 
about fourteen years teaching experience. 

To summarize the trends into one sentence, we are a global (trends 1, 4, and 
14) yet local (trends 3 and 5), expanding (trends 2 and 13) group of teachers 
who are in the process of becoming a profession (trends 8, 10, 11, 12, and 15) 
and setting our own agenda (trends 6, 7, and 9). 

For a breakdown of the answers see Appendix. For a more visual represen- 
tation of the results, the following pages present the trends seen as bar charts. 
Percentages have been rounded-off to the nearest whole number and, there- 
fore, the totals are not always exactly 100%. 

Additional Comments From Current JALT Leadership 

My third question is, what additional trends do JALT leaders perceive? The 
trends sui*vey contained one open question which asked respondents if there 
were any trends they felt had been overlooked and if there were, to please 
write them in a provided space. Following are the additional trends suggested 
by the individual respondents. 

The JALT leadership perceives a trend towards: 

1. Students learning more than one foreign language. 

2. Students learning languages in the host language country; participating in 
more overseas English study courses. 

3. Studying the similarities in Asian 0apan, Korea, Indonesia, China) learn- 
ers of English. 

4. Language teaching as a business. 

5. Using English to teach culture. 
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Trend i. As a knowledge-based field, we are becoming part of the 
global information network. 

no opinion disagree agree 

11 % 24 % 5 % 



m 



Trend 2. There is a trend toward growth in our field, 
no opinion disagree agree 

4 % 34 % 63 % 




Trend 3- The new Monhusho course guidelines will have a significant 
impact. 

no opinion disagree agree 

13 % 27 % 60 % 




Trend 4. There is a trend towards the establishment of U.S. colleges and 
universities in Japan and these schools will influence English teaching, 
no opinion disagree agree 

11 % 75 % 






Trend 5. There is a trend toward the JET, AET program having a greater 
influence on us. 

no opinion disagree agree 

2 % 58 % 41 % 




Trend 6. There is a trend towards teachers becoming more interested 
in classroom-based research. 

no opinion disagree agree 

4 % 22 % 74 % 




Trend 7. There is a trend toward the relationship between theory and 
practice becoming closer. 

no opinion disagree agree 

4 % 43 % 54 % 
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Trend 8. There is a trend toward feeling lost between two worlds 
because we don’t have a theory to make sense of or to justify our 
teaching. 

no opinion disagree agree 

6% 61% 34% 




Trend 9. There is a trend towards a focus on the learner as an 
individual, for example learner training and learner development, 
no opinion diasgree aarf 

4% 1 2% 84% 




Trend 10. There is a trend toward seeing teaching as unique and as 
having its own skills and knowledge. 

no opinion disagree agree 

8% 25% 67% 



Trend 11. There is an increasing awareness that we are a separate 
discipline and not a branch of some other discipline, for example 
linguistics or literature. 

no opinion disagree agr€ 

8 % 1 9 % 74 % 

Ml III I ill I I ll I IjlijjliiiinNiljiilHiiljliill!^ 



Trend 12. There is a trend toward becoming more concerned with the 
development of teaching credentials. 

no opinion disagree agree 

7% 8% 86% 




Trend 1^. There is a trend towards English nonnative speakers playing 
a more important role in the teaching of English. Already most EEL 
teachers in the world are nonnative speakers. 

no opinion disagree agree 

6% 14% 81% 
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Trend 14. There is a trend towards including human issues in our 
curriculum such as peace and environmental education, 
no opinion disagree agree 

6 % 1 5 % 80 % 




Trend 15> There is a trend toward psycholinguistics becoming more 
important for language teachers. 

no opinion disagree agree 

7 % 37 % 55 % 




6. Introducing English into the school curriculum at an earlier age. 

7. Children beginning to learn EFL at an earlier age. 

8. Junior high school and high school teachers being better speakers of 
English and therefore graduating better students. 

9. Presenting qualitative research under the guise of pseudo-scientific quan- 
titative research. 

10. Curriculum evaluation by both teachers and students increasing at the 
university level. 

11. Decentralized authoritarian control. 

12. Using part-time, non-tenured, non-professional instructors. 

13- Not opting for any one particular method, a trend back toward eclecticism. 

14. Looking too hard for trends (especially where Monhusho is involved). 

15. Teaching English as an international language (EIL). 

16. Moving back to grammar. 

17. Learning objectives which are more narrow than “communicative compe- 
tence,” e.g., ESP, translation courses, survival courses, English for aca- 
demic purposes. 

18. ESP/EST 

19. Teacher development. 

20. Testing. 

21. Downplaying the learning of English because the rest of the world is 
going to learn Japanese. 

22. Employing native English speakers more for their market value than for 
their effectiveness as teachers. 

23. Limiting instruction, skills, and content to the elements students will need 
to become efficient corporate components and heroic consumers. 

This list contains some trends that are cryptic and in need of explanation and 

clarification. In addition, all of these “additional trends” need additional support 
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such as references from the literature or empirical support from other sources. 

In addition to additional trends, some respondents included comments. 

Respondent Comments 

• There is tremendous variability in the definition of what a language teacher is. 

• Teachers should pay more attention to the relationship between CLT and 
University entrance examination; more study should be done in this field, 
especially experimental research. 

• Some of these trends 1 wish were happening but are not. 

• It is disappointing that many JETs in high schools can’t expect universities 
to change their exam structure. The government should take some measure 
to introduce aural tests in the national uniform exams. 

• I hire part time teachers for universities and I agree that better qualified ESL 
teachers in terms of degrees are available. An MA or Ph.D. in TESOL plus 
publications is almost required. 

• As a gaijin (foreigner) I circled 6 for many of the trends, but I believe that 
for the Japanese teachers it would be a 1 or a 2. 

Discussion 

One issue for discussion is who do the respondents of this survey represent 
and who do they not represent. Not every leader currently active in JALT was 
sent a suiwey and some that were sent a survey chose not to respond. The 
Japanese leadership is under represented. University teachers are adequately 
represented, but teachers in other institutions are not, and as a result their 
insights are lacking. Nevertheless, given these limitations, it is my experience 
in JALT that the group who responded to this survey do, in fact, reflect the 
JALT Executive Board as it is presently constituted and in that sense can be said 
to represent the thinking of the current JALT leadership. It remains for future 
suiweys to document change. For research into trends, both global and in 
Japan, to continue, it would probably be necessary for teacher groups such as 
JALT to officially support a larger-scale survey. 

Comments on the Survey 

This questionnaire survey was not field tested, reports no reliability coefficients, 
and offers no validation. For these reasons this article should be thought of as 
an interpretative essay reporting insights rather than research reporting general- 
izable results. There were two areas of comments by the respondents of the 
survey. Both areas are concerned with clarity. One is the issue of instalment 
clarity and the other is the issue of theoretical clarity. 

Respondents were asked to circle the number for each question that best 
showed their opinion. I should have stipulated in the directions to circle only 
one number because some respondents circled more than one number, thus 
invalidating their answer. For example, one respondent in answering trend ques- 
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lion numiDer two, wrote “short term” and circled number 3 and also wrote “long 
term” and circled number 6. Another indication that my directions were lacking 
in clarity in that some respondents answered by saying in their opinion the 
liieraiure indicated one position, but that their experience indicated another 
position. Some respondents seemed unclear about whether I meant them to 
answer as if they were in japan (which they were) or from some other place. 

In addition to a lack of clarity on how the survey instaictions were phrased, 
a lew respondents sensed a lack of conceptual clarity. One respondent wrote, 
"What I think or what teachers in general believe?” Another respondent com- 
plained that the sur\^ey was unclear because it did not specify the reference 
point as what each teacher personally believes, what native speaker EFL college 
teachers believe, what non-native speaker EFL teachers in Japanese public schools 
believe, or what the fresh gaijin in the JET program believes. 

The definition of what a trend is and is not needs to be made clearer. What 
constitutes a trend? How does one know when a trend exists? Does agreeing 
with a trend signify that one is acknowledging its existence or expressing 
approval? The,se and other questions need to be discussed and taken account 
of in future trend suiveys. 

When we look at the trends from the literature listed in this paper and con- 
sider the degree of agreement or disagreement of the JALT leadership with 
those trends, we see that the JALT leadership agrees with most of the trends 
with some important differences. JALT leadership group does not believe that 
IJ. S. colleges and universities are expanding and will have a significant impact 
on our teaching in Japan (trend 4), it does not yet see the influence of the AET/ 
JET jDrogram (trend 5), nor does it feel lost due to the lack of a comprehensive 
theory that can guide pedagogy (trend 8). In addition, the JALT leadership is not 
sure about there being a closing of the gap between theory and experience 
(trend 7). That being the case, it is not surprising that they do not experience 
p.sycholinguistics (trend 15) as being particularly helpful. 

Given that the topic of this collection of papers is on classroom based re- 
search, it would be helpful to look more closely at trend 6, that more teachers 
are becoming interested in classroom based research. As you will recall, there 
was 74% agreement with this trend. We will look at the background of this trend 
as well as some of the issues which are involved. 

Class room Resea rch 

The classroom teacher has traditionally been a marginal research participant. 
For at least the last 20 years, the defining centerpiece of ESL teacher education 
has been the teaching method taught in the methods course (Gaies, 1991; Nunan, 
1989). The methods course lacked training in classroom based research and 
tended to promote a method or methods for teaching (Grosse, 1991). The method 
(e.g, TPR, Silent Way, CLL) was often promoted by a teaching guru who viewed 
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the teacher as irrelevant at best or a contamination of the process at worst 
(Gaies, 1991). h was the teaching method that was seen as responsible for 
learning. The teacher’s job was to correctly administer the method and/or stay 
out of the way of the method as it connected with the students. 

The “classic’’ researcher has special training and special statistical tools with 
which to conduct research which the classroom teacher almost always does not 
have. Therefore, the role of the classroom teacher is to receive the research- 
This sets up a dichotomy of researcher as producer of knowledge and class- 
room teacher as consumer of knowledge (McDonough & McDonough, 1990). 
Because research is thus seen as a top-down arrangement coming from the 
researcher who is specially trained for the job, there is a mismatch between the 
role of the classroom instmctor and the ESL researcher which makes it difficult 
or impossible for the two parties to communicate. 

A new image is currently in the process of being formed for the role of the 
ESL teacher. This new image is multifaceted in that it includes both classroom 
practices as well as a research role. Genesee (1994) observed that the title 
“teacher-researcher” is part of this new image, but a mixed metaphor. On the 
“teacher” side of the metaphor, Genesee agreed that more teachers are currently 
involved in research than before and furthermore that this research is increas- 
ingly classroom based. On the “researcher” side, he expressed concern that 
teachers may be judged by criteria appropriate to researchers but not teachers. 
Allwright (1994) responded by suggesting that Genesee would be correct only if 
he accepts the classical definition of the researcher. Allwright pointed to the 
classroom and teacher pedagogical practices as the locus of an image of re- 
search more suited to teachers and the language classroom. McDonough 8c 
McDonough (1990) called this type of classroom research bottom-up research 
as opposed to the more traditional top-down research model. 

Gaies (199D aiticulated an image of the teacher as manager because in his view 
teaching is decision making. It is the nature and process of teacher decisions that 
ought to be looked at carefully in classroom research, especially as they promote 
change in classroom pedagogy and practices. Nunan (1989; 1996) said teachers 
should be monitors and researchers of their own teaching practices. To accom- 
plish this, teachers need special training in classroom research. Appropriate forms 
of research include action research and ethnograpliic research both of which look 
closely at what actually happens in classrooms as opposed to what teachers be- 
lieve happens. This view is shared by Grosse (1991, p. 44) who stated that a major 
challenge to teacher education is the “gap between what teachers and researchers 
think happens in the classroom and what actually goes on.” A major concern 
underlying this discussion of teacher as researcher is how we as classroom teach- 
ers initiate, promote, and manage change and innovation in our classrooms, in our 
institutions, and ultimately in the larger society. 
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Conclusion 

This paper began with the image of a train. The train was moving in unfamiliar 
territory and the question was asked, where are we now and where are we 
going? 1 will continue to use the pronoun “we” to indicate that the answer to 
this question affects the entire ESL/EFL teaching profession. 

The answers to this suivey seem contradictory. On the one hand, we be- 
lieve teacher training and credentials are important and we are interested in 
classroom research, but on the other hand we don’t feel a narrowing of the gap 
between practice and research. We believe non- native speaker teachers are 
becoming more important, but we suspect many non-native speaker teachers 
don’t share this opinion. We think the new Monhusho course guidelines will 
significantly impact us, but we don’t think the JET/AET program is having 
much i nil Lienee. 

Tlie tentative conclusion reached earlier in this paper was that we are a 
global/local group of teachers in the process of becoming a profession and 
setting our agenda. It is this last issue, setting our agenda, where I sense we are 
not clear and which I believe gives rise to our contradictory responses. At the 
moment we are getting ready for the agenda task by deciding what issues we 
think are important and preparing ourselves for the research task ahead. We are 
not sure though where we want to go or how to get there, but we sense that as 
classroom teachers it is up to us to make those decisions and preparations and 
not ask others to do it for us. Small wonder that we are a bit confused. 

You are now back on the train. You finish your class and look out the win- 
dow as the train approaches the station. As the train pulls in you notice a sign 
which announces the name of the station. The sign says, Welcome To Getting 
Ready. You get off the train and think to yourself, “Tliis must be the place.” 
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Appendix: Survey Results 

Percent Percent 

No opinion Positive Response 

Tend to disagree Tend to agree 

Trend 0 1 2 3 4 5 6 



Trend 1 


.11 


.02 


.07 


.15 


.15 


.30 


.20 


Trend 2 


.04 


,02 


.1 


.22 


.22 


.29 


.12 


Trend 3 


.13 


.05 


.07 


.15 


.29 


.22 


.09 


Trend 4 


.11 


.22 


.38 


.15 


.11 


.04 


.00 


Trend 5 


.02 


.13 


.25 


.20 


.15 


.22 


.04 


Trend 6 


.04 


.04 


.07 


.11 


.20 


.39 


.15 


Trend 7 


.04 


,00 


.17 


.26 


.22 


.26 


.06 


Trend 8 


.06 


.22 


.22 


.17 


.13 


.19 


.02 


Trend 9 


.04 


.00 


.06 


.06 


.28 


.39 


.17 


Trend 10 


.08 


,00 


.13 


.12 


.17 


.33 


.17 


Trend 11 


.08 


.04 


.11 


.04 


.25 


.26 


.23 


Trend 12 


.07 


.00 


.06 


.02 


.15 


.43 


.28 


Trend 13 


.06 


,00 


.04 


.10 


.24 


.39 


.18 


Trend 14 


.06 


.00 


.09 


.06 


.26 


.35 


.19 


Trend 15 


.07 


.04 


.09 


.24 


.31 


.17 


.07 
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A common way to do research is to plunge straight in, collect a vast amount of 
data, and tlien: disaster! You quickly panic as you struggle to find a way to 
process all the collected information. 

It is far better to approacli research in a systematic way. No builder would start 
construction witliout having first drawn up a set of architectural plans. In the same 
way, a research proposal is a modest document of one to four pages which shows 
concisely what die research topic is and how you are going to carry out the 
research. Having a properly written researcli proposal helps ensure that all parts of 
the research projea have been carefully thought about before the research begins. 
Without careful consideration beforehand, you run the risk of coming up against 
serious problems diat can result in a lot of time wasting and frustration. 

Sometimes a research proposal is compulsory. That is the case if you are 
doing research in the hope of getting an academic qualification such as a 
Masters or Doctoral degree. Research is often funded by foundations or gov- 
ernment agencies. Such bodies will require a research proposal before approv- 
ing the research or granting any funds. But even when it is not mandatory, it 
still makes sense to do a research proposal as part of the process of planning a 
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research project. Not only will this alert you to potential problems, but you will 
then have a handy document which you can use when you consult other, more 
experienced people for advice on how to solve those problems. With your 
proposal to work from, your more experienced colleagues are able to give you 
better advice on frequently encountered difficulties such as analyzing statistics 
by computer, and choosing suitable subjects. 

For several reasons then, writing a research proposal can be an important 
and useful step in die process of planning and implementing a research study. 
But, what exactly is a research proposal and how does one go about writing 
one? What follows is an outline to help in the writing of a proposal. Each step 
is explained and an example taken from the proposal written by one of the 
authors (see Appendix). The completed research study was briefly discussed 
in Ellis (1994). 

The form that this research proposal outline takes is similar to that of the 
final document written after the data has been collected and analyzed. This 
will help save time and make the job of writing the final report easier. 

Title 

A. Introduction 

1. Context — description of topic or problem 

2. Review of literature 

3. Research questions or hypotheses, or both 

B. Methodology 

1. Subjects 

2. Design 

3. Data collection 

C. Analysis 

1. Data organization 

2. Statistical procedures 

D. Significance of Study 



Title 

Tlie first diing you need is a working tide for your research. Research ddes have 
am the gamut from lengthy, arcane stuffy ddes to instant attendon-getting quirky 
ddes such as “Say it Again Sam” or “The Chicago Which Hunt.” Our advice is to 
steer a middle way. Have a dde which gives the reader a clear idea of die topic but 
which is also brief. One way to do this is to use a short statement followed by a 
colon. After die colon can be a lengthier delineadon of the topic. Tliis is the 
approach taken by White (1992) in deciding on the dde, “Teachers’ Quesdons: 
Form, Funcdon and Interacdon — A Study of Two Teachers.” 

36 
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Introduction 

Tlie research proposal begins with an introduction. Usually tliis is not lal^led 
specifically “Introduction.” Tliis convention is also followed by journal articles. 
Check accounts of research in such journals as TESOL Quarterly and JALTJoumal 
As indicated in tlie above outline, tlie introduction has three parts. The first part 
gives die reader an idea of die focus of the research. The second part attempts to 
link the research issue with existing information. Having reviewed the existing 
litemture — die airrent body of information relevant to your topic — ^you need to 
establish why your research is important and timely. One way is to show that there 
is a lack of information relating to your topic or alternatively, that you can add a 
significant and different perspective. Having established a need for your research, 
you can then state your research questions or research hypotheses, or both, 

Nunan (1992) stated that the minimum requirements for an activity to be 
considered research are: (a) a question; (b) data; and (c) analysis and interpre- 
tation. You cannot begin to plan or carry out research until you have a research 
question. The first step is to find a topic or area that interests you. This can 
come from your experience as a classroom teacher or language learner or from 
your own reading. Reading widely in academic journals can give you ideas for 
research. If you are interested in second language acquisition it is a good idea 
to read accounts of how children learn their mother tongue, or journals in 
fields other than language learning. For one piece of research we read journals 
dealing with social psychology and found ideas for a study dealing with the 
different factors that ’cause people to adjust their speech for different partners. 
If you are really in pursuit of ideas for research then the conclusions of doc- 
toral theses are good hunting grounds because it is customary in the closing of 
a thesis to mention questions that the current study did not answer and to give 
recommendations for further study. 

Having found an area that interests you, you will need to form a research 
question. Good research questions meet two criteria: (a) the question merits 
. answering and is worth seeking an answer for, and (b) the question can, in 
fact, be answered. 

In the research on teachers’ questions, we started our proposal with the 
following statement to define what the research was about: “The kinds of 
questions teachers ask in classrooms has increasingly attracted the attention of 
researchers in recent years.” We then did a brief survey of the existing research 
dealing with teachers’ questions. 

The Literature Review 

The literature review has several functions: (a) it gives the background to your 
proposed research, (b) it shows that you are competent in that you have an 
understanding of what is currently known, and (c) it provides a reason for your 
research. 
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The review section usually ends witli tlie author discussing the implications 
arising from the existing literature. Here is the place to point out gaps in exist- 
ing knowledge on the subject, or the need to look at the issues from a fresh 
perspective. In this way, you provide justification for your proposed study, and 
demonstrate its importance. 

In our proposal, the review of the literature pointed out that recent research 
on teachers’ questions had moved towards experimental and quantitative re- 
search with reports containing a lot of statistical analysis but fewer examples of 
actual classroom discourse when compared to previous research in LI settings. 
We mentioned that the study proposed “to fill this gap by not only analyzing 
the frequency of questions according to form and function but also: , , including] 
the additional dimension of discourse analysis.” 

Questions or Hypotheses 

This section states your carefully tliought up and worded hypothesis. In our 
research on teachers’ questions we asked three questions: 

1. What type of questions do teachers ask? 

2. Which types of questions result in tlie longest student response? 

3. Will there be any difference in the types of questions and responses accord- 
ing to the proficiency level of the class? 

In addition to research questions, we also stated the object of our research in 
the form of a hypothesis. If your purpose is to acquire information — for ex- 
ample, the order in which second language learners learn morphemes — then 
the research area you will tackle is put as a research question. If, however, you 
are testing a theoiy, tlien you should state tlie object of your research as a 
hypothesis. The difference is that the answer to a hypothesis is a simple yes or 
no. The theory was either proved or it was not. With a research question, 
however, you can have a much wider range of answers. In our paper we 
included three hypotheses: 

1. Teachers will ask more display questions than referential questions. 

2. Learner responses to referential questions will be longer than the response 
to display questions. 

3. A greater number of referential questions will be accompanied by a greater 
number of confirmation checks and clarification requests by the teacher. 

Method 

In this section of the proposal you include a description of how the research 
will be carried out and the method you will adopt to answer your research 
questions or hypotheses. You include who the subjects will be, the kind of 
research it is and how you will collect the data. 
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One test of a good metliodology section is if after reading it somebody 
could proceed and carry out the research based solely on the information you 
have given. A straightforward way to tackle this section is to have three sub- 
categories: (a) subjects, (b) design, and (c) data collection. 

In our proposal on questions, we started tliis section by explaining tliat “the 
subjects for tliis study will be 43 adult students attending the in-company language 
training program of a major multi-national company.” Next we profiled die teach- 
ers who would be die focus of die saidy. At diis point we also explained the basis 
for choosing diese two teachers. “Tlie two teachers were selected on the basis of 
similar sex, nationality, and teacliing experience. They also had access to classes at 
comparative levels and could teach die same lesson material to their classes.” 

Design 

This section is die place to describe the type of research you are going to cany 
out. In our report we handled it diis way: “This is intended to be descriptive 
research and will use naturally existing classes for the purpose of data collec- 
tion. It will be a one-shot design in diat each class will be observed only once.” 

Data Collection 

In this section should include a detailed and accurate account of how you will 
go about collecting the data you need. In addition, this section should mention 
what equipment or materials will be used. Questionnaires are often listed here 
if the research in question is of a survey nature. The information in this section 
should be so clear diat the reader can not only visualize the process of how 
your are going to do your research but would also have sufficient information 
to replicate your research. It should also be clear to the reader that die steps 
you have outlined are sufficient to answer the research questions mentioned. 
In our research proposal we mentioned that we would take four audio record- 
ings of one hour each, and we listed the teachers and the classes that would be 
recorded. We also mentioned the newspaper article and the material that was 
to be used in the classes. 

Analysis 

This part of the proposal has typically two parts: (a) data oiganization, and (b) 
statistical procedures. Tliis section focuses on the question of what you are going 
to do widi all die data when you have colleaed it. It can be frightening to collect 
thousands of pieces of data and not know what to do with the information. Tliink- 
ing in advance and putting somediing down in writing before you start can save a 
lot of faistration and wasted effort. For our paper on teachers’ questions we tracked 
down, in die literartire, various scliemes for categorizing questions. Tlie resulting 
classification scheme, a taxonomy, was laigely what we had read in books and 
journals but was also modified to better suit the aims of the research. If you are 
going to use charts and tables you can also mention them at this point. 





44 



Martin White & Diane White 



Statistical Procedures 

This part can be the most daunting of the whole research project. Luckily, if 
you need it, you can get help to steer you through the maze of deciding 
whether you need to use ANOVA, Pearson Correlation, or Chi-square, We found 
what statistics would help analyze our data to prove or disprove our hypoth- 
eses and were able to write, “Statistical frequencies obtained will involve 2- 
way Chi-square,” Not all data will need a computer to process tlie numbers. 
For our paper on teachers’ questions we did a lot of counting by hand. That 
involved adding up the number of times a student or teacher took a turn in 
speaking, and calculating the mean number of words in each student response. 

Significance 

This section is the place for you to state the importance of your research study. 
In it you explain the contribution it will make to linguistic science or whatever 
field you have chosen to write about. In the proposal on teachers’ questions, we 
set about the task in this way. 

To the best of our knowledge no study has been made using Well’s Discourse Role 
and Commodity Network for an analysis of the exchange structure of classroom 
discourse in an adult EFL setting. The effect of different proficiency levels has also 
not yet been researched. The proposed study outlined above is intended to con- 
tribute to a greater understanding of the role of teachers’ questions and to provide 
more illustrative samples of actual classroom discourse. 

It is said that there is nothing so practical as a good theory. Tlie research you are 
proposing to carry out may have the potential to change peoples’ lives — or at 
least how they teach and learn languages. If the results of your research have 
this potential here is the place to make the announcement. 



When you have written your research proposal, you should review it, possibly 
with the help of a colleague. One of the authors, who teaches research writing 
classes at a Japanese university, has found that the biggest problems tliat novice 
researchers have is defining the research question. As Nunan (1995, personal 
communication) has pointed out, this can also be a problem for experienced 
researchers. The problem is that often would-be researchers are unclear about 
knowing what they actually want to study. It is crucial that a beginning re- 
searcher has a clear understanding of what they want to research. If you have 
carefully thought out what you want to study and defined tliis in a suitable 
research question then the research has the greatest chance of going very 
smoothly. Read over your research proposal and make sure that it is clear what 
your study acaially proposes to do. Make sure too, that the topic is worth 
investigating. Next check that the topic is sufficiently focused. Another major 
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problem with many initial proposals is tliat the scope of the research is too 
broad. This leads to problems of data collection and analysis and if it is too 
broad tlien it may be impossible for the research to lead to any conclusive 
findings. Our advice would be to keep tlie focus of enquiry relatively simple. 

When reviewing the proposal also examine carefully the choice of subjects 
and the procedures tliat will be used. Make sure that you have carefully consid- 
ered tlie relevant variables. For example, if you are comparing two groups and 
one group performs better using your material, can you safely say it was solely 
due to die effectiveness of the materials? Perhaps die group that did better was 
a liigher level group or diose students usually perfomi better since their teacher 
is more enthusiastic and diey are therefore more modvated to learn English. 
Also make sure you can jusdfy your use of subjects. It is not enough to reply 
diat it is just convenient to use this set of students or that it is easier to get 
permission from a certain teacher. 

Anodier problem frequently encountered is the difficulty of finding relevant 
background material. The following journals are a good source of articles diat 
cover teaching, learning, and applied linguistics. Applied Linguistics, ELT Jour- 
nal, English Teaching Forum, JALT Journal, Language Learning, Studies in Sec- 
ond Language Learning, System, TESOL Quarterly. 

Researchers interested in using the computer to do researcli can access ERIC 
(Educational Resources Information Center). ERIC is an abstract service that lists 
important journal articles and conference papers according to various categories. 



Conclusion 

Writing a research proposal is a preparatory stage in the cycle of doing a re- 
search study. Seliger and Shohamy (1989) stated that there are four phases to 
die preparatory stage of any research study. 

Phase 1. Formulating the general question. Possible sources are the experience 
and interests of the teacher/researcher, other research on second lan- 
guage acquisition or sources outside second language acquisition. 

Phase 2. Focusing the question. Is the question both feasible and important^ 

Phase 3. Deciding on an objective. Description of research procedures and 
formulating research questions. 

Phase 4. Formulating die research plan. 

The research proposal is really a plan, a step-by-step guide to doing research. 
It will help ensure that you have given sufficient thought to all the cmcial areas 
of research, such as, die basic research design, data collection and analysis of 
die results. It will also help you to see and avoid many potential problems. In 
addition to helping give shape to the planning process, it also makes the final 
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writing up of the research easier as you already have the main headings for 
each part of tlie final research report. Once you have that done, you are ready 
for publication. In Japan two widely read journals that publish research are the 
JALT Journal and The Language Teacher. More and more teachers are carrying 
out their own research, particularly classroom-centered research. By following 
a step-by-step plan such as that outlined above even those not trained in re- 
search or statistical procedures could start a research project and make a valu- 
able contribution. 
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Appendix: A Research Proposal 
Title 

Teachers’ Questions: Form, Function and Interaction — ^A Study of Two Teachers 



Introduction 

Tlie kinds of questions teachers ask in classrooms has increasingly attracted the attention 
of researchers in recent years. Studies of teachers' questions (Long and Sato, 1983; Brock, 
1986; Long & Crookes, 1986) have analyzed the frequency of question types according 
to function, and in particular have sought to discover whether teachers ask more display 
questions than referential questions. Tlie results indicate that in ESL classrooms teachers 
ask too many questions (Wliite & Lightbown, 1984) and that these questions are typically 
display questions (Long & Sato, 1983). Studies in teachers’ questions have also concen- 
trated on identifying tlie effects of tlie use of comprehension checks, confirmation checks, 
and clarification requests. Such modifications to the interactional structure of conversa- 
tion, it is claimed, facilitate second language acquisition. Earlier studies of teachers' 
questions featured school classrooms in an LI setting. Studies such as Barnes, (1969) and 
Mehan, (1978) are informative and illustrated with extracts of classroom discourse. Later 
research, however, focusing on the ESL classroom has moved in the direction of quanti- 
tative research and contains almost no examples of recorded classroom discourse. In 
addition, studies such as Long and Sato (1983) and Brock (1986) have been experimental 
in design and have grouped students especially for the experiment or given advice to the 
teachers on how to teach the lesson. 
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The proposed study intends to fill this gap by not only analyzing the frequency of 

questions according to form and function, but also by including the additional dimen- 
sion of discourse analysis. 

The following research questions will be addressed: 

1. What types of questions do teachers ask? 

2. Which types of questions result in the longest student response? 

3. Will there be any difference in the types of questions and responses according to the 
proficiency level of the class? 

In addition, the following three research hypotheses, based on Brock (1986) will be 

tested: 

1. Teachers will ask more display questions than referential questions. 

2. Learner responses to referential questions will be longer and syntactically more com- 
plex than the response to display questions. 

3. A greater number of referential questions will be accompanied by a greater number 
of confirmation checks and clarification requests by the teacher. 



Method 



Subjects 

The subjects for this study will be 43 adult students attending an in-company lan- 
guage training program at a major Japanese multinational company, for four hours a 
week. In terms of proficiency as measured by TOEIC scores the higher class has an 
average score of 711 and the lower level class has an average TOEIC score of 424. All the 
students have had at least five years of learning English either at school, university, or in 
company classes. Also serving as subjects for the study are two teachers, both female 
and British. Both teachers, have the RSA Certificate in TEFL and one teacher also has the 
RSA Diploma and a Postgraduate Diploma in EFL and is presently writing a thesis to 
complete her master’s. The other teacher is in the process of earning tlie RSA Diploma in 
EFL. The teachers were selected on the basis of similar sex, nationality, and teaching 
experience. They also had access to classes at comparative levels and could teach the 
same lesson material to their classes. Tlie students will be grouped in their already 
existing classes and taught by their normally scheduled teacher. 

Design 

This is intended to be descriptive research and will use naturally existing classes for the 
purpose of data collection. It will be a one-shot design in that each class will be ob- 
served only once. 

Data Collection 

Audio recordings of one hour duration will be made from four classes in total: two from 
a low proficiency class and two from a high proficiency class. Both teachers will teach 
one low proficiency class and high proficiency class. Tlie teachers will use the same 
materia! for classes of a similar level. The higher level class will use an authentic, 
unsimplified article on illegitimacy taken from the British newspaper. The Daily Tele- 
graph. Tlie material used for the lower proficiency classes will be Unit 11 of Coast to 
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Coast (Harmer, 1987), a general English textbcK)k. Tlie teachers will be provided with a 
small professional tape recorder to place on the table. These recordings will be tran- 
scribed and coded for the frequency of question types. 

Analysis 

Taxonomy for the form of teachers’ questions: 

Wh-questions, Yes/No questions, Alternative questions, Rlietorical Function tax- 
onomy (modified version of Long and Sato, 1983). 

Display questions — form-focused/content focused/repetitions. 

Referential questions — open/closed/repetitions. 

Rhetorical/expressive/ repetitions. 

Comprehension checks, confirmation checks, clarification requests. 

Tlie number of turns per student and teacher and mean number of words for student 
responses after given type of question will be calculated. Statistical analysis of frequen- 
cies obtained will involve 2-way Chi-square. To examine the discourse role of questions, 
the system shown in Wells (1979) will be used. 



To tlie best of our knowledge no study has been made using Wells’s discourse role 
and Commodity network for an analysis of the exchange structure of classroom dis- 
course in an adult EFL setting. Tlie effect of different proficiency levels on teacher’s 
questioning behavior has also not yet been researched. Tlie proposed study outlined 
above is intended to contribute to a greater understanding of the role of teachers’ ques- 
tions and to provide more illustrative samples of actual classroom discourse. According 
to Long and Sato (1983), their results show that communicative use of the target lan- 
guage makes up only a minor part of typical classroom activities. Is the ESL classroom 
really so bereft of possibilities for genuine communication? It is the purpose of the study 
outlined in this proposal to investigate the truth of such an assertion. 
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Literature Reviews: 
Obtaining Perspective 



Paul Riley 

Oxford University Press 



“Would you tell me, please, which way I ought to go from here?” 

“That depends a good deal on where you want to get to,” said tlie Cat. 

“I don’t much care where — ” said Alice. 

“Then it doesn’t matter which way you go,” said the Cat. 

“ Lewis Carroll, Alice in Wonderland 

The question central to all research is, “Wliich way should I go from here?” 
Response to this question requires background knowledge to broaden the field 
of inquiry and then further research to narrow tlie parameters and frame the 
research question. This paper looks at the literature review — tlie researcher’s 
attempt at gathering tlie necessary background information to figure out where 
one is and, subsequently, where one is going. It is aimed at teachers with little 
or no research experience and endeavors to give step-by-step guidelines for 
conducting and using a review of Second Language Acquisition (SLA) literature. 



What is a Literature Review? 

A literature review or litemture search is essential for any research project. It usu- 
ally takes place after the researcher has decided on an area of research interest, but 
before the research question has been formulated. Often there is a second and 
even tliird review later in tlie research process to expand or tighten up tlie scope 
of tlie project. Tlie literature review entails the researcher locating, in journals and 
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books, research articles relevant to liis or her research interests. After collecting a 
wide and thorough sample of pertinent articles, tlie researcher must read, syntlie- 
size and report on the infonnation uncovered during the survey. 

Brown (1988, p. 46) wrote that a literature review should “provide tlie back- 
ground or rationale for tlie study, a demonstration of how previous research is 
related to tlie study, and a framework for viewing tlie study.” Nunan (1992, p. 2l6) 
agreed tliat, “the literartire review, if carried out systematically, will acquaint you 
witli previous work in tlie field, and should alert you to potential pitfalls in the 
chosen area.” He added tlie caution that, “you may come across a study wliich 
answers tlie veiy question you are proposing to investigate.” Seliger and Shohamy 
(1989, p. 65) referred to literature reviews as “contextualizing tlie research.” They 
argued tliat a suivey of tlie literature is undertaken for the seemingly contradictory 
puiposes of broadening tlie perspective of the project and narrowing down the 
topic, “because in research tliere is a need to botli expand the perspective and to 
narrow it down in order to aiTive at a workable research question.” 

Although tlie terms and definitions differ slightly from researcher to researcher, 
there is agreement that a review of tlie literature is necessary for meaningful 
research. In fact, without a proper survey of tlie literature, tliere could be trouble 
formulating the research question. The intention of a review is not to boldly go 
where no one has gone before, but to follow the paths others have pioneered 
before striking out into the uncharted wilderness. Undertaken in the pre-re- 
search question stage, it helps to build a foundation for the project and ensures 
that the researcher does not reinvent the wheel. Performed in the middle of a 
project, it broadens the researcher’s perspective, and suggests new avenues for 
inquiry. Conversely, late in tlie researdi process it helps to focus die evaluation 
of data, and further clarify die research question. 



How to Begin 

It is not my intention to grapple with probably the most difficult stage of research: 
research design and formulating die research question. Suffice it to say, die first 
diing to be determined, at least in a general sense, is the research idea. Tliis is 
usually a difficult task for die novice and requires curiosity, observation, and read- 
ing on the part of the potential researcher (see Griffee, 1994). But, as Seliger and 
Shohamy (1989) pointed out, it becomes easier widi experience because “research 
is cyclical. It is an ongoing activity which is never totally completed because each 
piece of research raises additional questions for more research.” 

When die research idea has been decided, the first diing to be done is a general 
literature survey to determine die liistory (or lack diereof) of die proposed topic. If 
similar studies have been carried out, the researcher should use die findings to 
determine the present state of die field and decide if diere are any unanswered 
questions. However, if there have been no similar saidies, die researcher must 
ascertain die reasons why and determine the feasibility of conducting such re- 
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search. At tliis point die researcher will need to define concepts, determine the 
type of study (e.g. heuristic or deductive), and formulate the research question. 



Obtaining References 

Tliere are many ways to begin a general survey. The best is to speak with an 
author or researcher who has primary knowledge in the area of interest These 
people gather at large language conferences, such as the JALT National Conven- 
tion or the TESOL Annual Convention, and present papers in their special fields. 
It is usually possible to attend these presentations and with persistence meet the 
speaker and ask for advice. As well, these speakers often distribute bibliogra- 
phies or recommend articles for further study. 

Anodier source of material that can be found at these conventions are the 
publishers themselves — ^major publishers will all have displays manned by knowl- 
edgeable representatives. Aldiough biased towards their own products, the rep- 
resentatives will suggest appropriate books, which have useful information and 
unbiased bibliographies. A third option is to ask other teachers in attendance at 
the conference for help. By asking enough people, the enthusiastic novice is 
sure to find someone who can point her or him in the right direction. 

Technology offers die opportunity to communicate with fellow researchers 
dirough die Internet. Anyone with access to a computer with a modem can 
send and receive information from anywhere in the world quickly and eco- 
nomically. According to Busch (1994, p. 17) a good place to contact others in 
the SLA field is TESL-L, a computer bulletin board sponsored by the Interna- 
tional English Institute of Hunter College, which is located in New York City. 
Over 1,400 ESL teachers and academics from 48 countries use the bulletin board 
to communicate on topics concerning second language acquisition. TESL-L can 
be joined by sending the message “Sub TESL-L <first name last name>” to 
<listserv@cunyvni.bitnet> or <listserv@cunyvm.cuny.edu>. 

A different way to get started is by reading a general applied linguistics book 
(diey usually contain massive literature reviews), that gives an overview of the 
issues in die second language acquisition field. A couple of the best are The 
Study of Second Language Acquisition, Ellis (1994) and An Introduction to Sec- 
ond Language Research, Larsen-Freemen and Long (1991). These are compre- 
hensive studies of the SLA field, and are invaluable as an introduction and 
research reference. As well, both have extensive bibliographies. 

Of course, the most tried and true method of obtaining suitable references is to 
go to a library and start digging around. Unfortunately, because teachers of English 
as a foreign language (EFL) are often living and working in places where suitable 
libraries are scarce, this kind of research can be difficult or impossible. One option 
is to approacli university libraries and ask for permission to use their facilities. 
Usually, international universities or local campuses of foreign (especially Ameri- 
can) universities have the resources for second language research and will allow 
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use of their library for a fee. Privileges are restricted, however, and often a letter of 
introduction from a home library or a guarantor is required as part of the applica- 
tion. For more information on how to gain library access, read “Using Japanese 
Libraries to Do Second Language Research” (Busch, 1994), or call university librar- 
ies directly to find out about their visitor policies. 

Those lucky enough to have a good library nearby should first check if the 
library is connected to ERIC (the Educational Resources Information Clearing- 
house). ERIC is a computer database which lists documents, journals, and serial 
publications on different topics related to education. Researchers using ERIC 
enter-in a field of descriptors — defining the parameters of tlie search — and re- 
ceive a list, including abstracts, of citations that meet those conditions. Any 
documents or journal articles that seem relevant can be ordered for a fee. For 
more details, read “Research in Japan” (Baskin, 1994), which gives an excellent 
overview of using ERIC and ordering documents from within Japan. 

If ERIC is not an option, the best place to start is recent copies of any of the 
major journals in the second language field: Applied LinguisticSy Language Learn- 
ing, Second Language Research, Studies in Second Language Acquisition, and TESOL 
Quarterly. These journals are all widely available, and many print an index — 
usuaUy in the first or last issue of the year — of all articles received in the previous 
12 months. Scanning the titles in the index should offer a number of possibilities 
which can be assessed by finding the article and reading the abstract. 



Extracting Relevant Information 

Though there are many ways to obtain references, there is only one way to 
extract the relevant information, and that is to read. After collecting a stack of 
articles or list of references, the researcher must separate the wheat from the 
chaff by reading the abstracts to decide if the material is germane to the re- 
search interest and worth reading. Again, the novice may find this difficult, but 
as one’s knowledge of SLA broadens, articles can be assessed more quickly. 
Criteria to consider when determining the importance of material include the 
relationship. of the material to the proposed research, the source of the material, 
and recency of publication. Of course, articles with a direct relationship to the 
proposed research will be more important than those with only a secondary or 
tertiary connection. Likewise, primary information from well known sources is 
preferable to second-hand reports or information from unknown or unpub- 
lished sources. Finally, researchers should try to review the most up-to-date 
literature available. It is necessary to obtain the most recent material to ensure 
coverage of the latest developments in the area of interest. 

It is a good idea to devise a note-taking system to oiganize the reading of the 
literature. Seliger and Shohamy recommend using a computer database or index 
cards, and I prefer a small notebook, but the information that needs to be recorded 
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remains constant regardless of tlie medium. For each article the researcher must 
copy down bibliographical information, and should identify and summarize the 
main arguments or findings. It is important to point out the relevance of tlie mate- 
rial to tlie researcher's own interests and write down any ideas on how tlie infor- 
mation fits into a concepaial framework of the field. Tlie researclier may also want 
to identify and mark possible quotations to be used when reporting on tlie review. 
It is helpful to organize tliese siuiimaries in some order: alphabetic, by importance, 
or chronologically. Irrelevant materials should be discarded. 

Reading tlie articles is time consuming and can become overwhelming, as each 
article offers further areas for inquiry and the research process becomes a series of 
unending trails. Tliis broadening of perspective often leads into other disciplines 
(Sociology, Psychology, Education, Linguistics). Brumfit (1995, p. 37) encouraged 
tliat tendency: "Research needs to be interdisciplinary, because our reactions to the 
people with whom we work cannot be constrained by single disciplinary perspec- 
tives.” Tlie question then becomes, "How much is enough?” 

Literamre reviews are conducted for different purposes, each with specific re- 
quirements. If the researcher is writing for a pedagogical journal like The Language 
Teacher, a brief background check witli no more than a half dozen to a dozen 
sources is usually sufficient. However, research oriented periodicals like tlie JALT 
Journal, TESOL Quarterly, or Applied Linguistics require more substantial investi- 
gation. Bibliogmphies in these journals are often a few pages long. Tlie number of 
relevant sources should act as a guide to the inexperienced researcher. Too many 
sources may mean the topic is too broad and needs to be narrowed down. Con- 
versely, too few references may indicate a need to expand the scope of tlie le- 
searcli, or tlie need to work harder to dig up relevant material. 



Writing the Review 

As stated earlier, while reading and summarizing the various articles, the re- 
searcher needs to simultaneously organize the extracted information, and try to 
fit it into its place in the survey. A literature review should not be a discussion of 
one article after another — one paragraph per study. Instead, the researcher should 
try to develop a framework in which a conceptual order is imposed. This con- 
ceptual framework should identify and define the important issues or variables 
in the field, and the researcher should organize the review to discuss each one 
in the context of the literature. What should emerge is a clear picture of the 
history and current state of the research area, what Brown (1988, p. 46) called, 
"a framework for viewing the study.” He went on to add that “an author who is 
broaching an unexplored area of research cannot cite previous works [but] 
should at least explain the route by which tliis new area was reached.” 

Once the past and present have been discussed, it is time to consider the 
fuaire. The researcher must state the rationale of his or her own research 
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project, and consider the significance of the research for language pedagogy. 
This rationale could contain an evaluation of earlier research, and should in- 
clude a statement of purpose to justify the proposed research project. Often 
here is where the survey will end. 

In graduate scliool or teaclier training courses, the literature review forms the 
foundation for mucli of the work, and the researcher is a student who is assigned 
a survey to get a grounding in SLA. If the researcher is conducting the review to 
support liis or her own research projea, there should be enough infomiation to 
shape tlie researcli idea and focus it into a research question. As the project contin- 
ues, tlie researcher will probably need to periodically refer back to tlie survey for 
support. It is also necessary to add to and edit tlie review as the research matures, 
tlie researcliefs perspective changes, and the project focus narrows. 



Summary 

Tliis paper endeavored to give the novice researcher some of the guidance and 
information needed to conduct a survey of SLA literature. The literature review is 
tlie most fundamental type of research. It forms the basis for much of the work at 
graduate school, and is essential for teacher training courses. It provides tlie neces- 
sary background to frame research projects and leads researchers towards their 
research goals. Performed early in a research project, it helps build tlie foundation 
on which to base hiitlier inquiry. As the research progresses, the literature review 
broadens the perspective of the study and helps to sliape the research interest into 
a focused idea. The review requires the researcher to be a sleutli and a critic. It 
makes teachers look at tlie theoretical underpinnings of their profession and sub- 
sequently leads to greater understanding and professional development. 
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Tills paper introduces some of the overarching issues in second language re- 
search. They are issues wliich must be addressed before conducting a study so 
tliat tlie researcher can avoid conceptual pitfalls that may cripple the study later 
on. The discussion will begin with the considerations involved in sampling a 
group, or groups, of subjects to be used in a study. Next, the different typ>es of 
variables tliat researchers define in a study will be covered. Then, some of the 
research designs tliat can be used in second language studies will be explored. 
In addition, tlie factors whicli may jeopardize the internal and external validity 
of language studies are covered. Finally, the ethical issues involved in collecting 
data, conducting research, and reporting the results will be discussed. 



Sampling 

In language studies, it is often necessary to use sampling techniques. To under- 
stand why such teclmiques are necessary, it is first imp>ortant to grasp the differ- 
ence between a population and a sample. In research, a population can usually be 
defined as the entire group of language speakers or learners that the researcher 
wants to smdy. Unfortunately, few researcliers have the resources to study, for 
example, tlie entii'e population of ESL students studying ESL in American universi- 
ties, or the entire population of EFL students in the world, or even all of the male 
chemistry students from Gemiany who are studying in the United States. As a 
result, most researchers prefer to use a sample, that is, a subgroup of the students 
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representative of the given population. By using a sample, data can be practically 
and effectively collected, sorted, and organized. There are two basic strategies that 
are generally used in language studies for selecting samples from populations. 
These strategies are called random sampling and stratified random sampling. For 
both approaclies, die purpose is to create an accurate sample, or subgroup, which 
can be said to be representative of the population as a whole. 

Alternative Sampling Strategies 

The underlying principle in random sampling is that each individual member of 
the population must have an equal chance of being selected into the sample. 
Three steps can be used to insure such equality of chance: 

1. Clearly identify the population in which the researcher is interested. 

2. Assign an identification number to each member of the population. 

3. Choose the subjects for the sample on the basis of a table of random numbers. 

A table of random numbers is a list of numbers, usually generated by a com- 
puter, that contains no systematic patterns. Most introductory statistics books con- 
tain such a list (for example, see Appendix A in Shavelson, 1981). Using a table of 
random numbers leaves the choices of who will be included in the sample up to a 
dispassionate and random table of numbers, rather than up to the researcher who 
may have subtle biases (conscious or unconscious) that could affect the results of 
the study. Once a large enough number of subjects is randomly selected, the 
resulting random sample can be assumed to be representative of tlie entire popu- 
lation from which they were drawn (Brown, 1988, pp. 111-113). 

Other, more readily available techniques can be used to obtain a random 
sample. For example, the researcher might like to pull numbers out of a hat, use 
a deck of cards, or repeatedly throw a pair of dice in selecting subjects for a 
sample. Any technique wherein each member of the population has an equal 
chance of being selected, thereby ruling out biases on the part of the researcher, 
will be acceptable for random sampling, whether the sampling be for selecting 
subjects from a population for inclusion in a study, or for separating tliem into 
subgroups witliin the study itself. 

Another strategy that is sometimes used in language studies is called strati- 
fied random sampling. In this case, four steps are usually used: 

1. Clearly identify the population in which the researcher is interested. 

2. Identify the salient characteristics of the population (called strata). 

3. Randomly select members from each of the strata in the population (using a 
table of random numbers or other techniques described above). 

4. Check to insure that the resulting sample has about the same proportions of 
each characteristic as the original population. 

For instance, in tlie population of all ESL students studying at the University of 
^Hawaii at Manoa (UHM), it might be useful to identify subgroups, or strata. 




52 



Designing a Language Study 



57 



witliin the population based on the following characteristics: gender (male or 
female); country of origin; native language; academic status (graduate, under- 
graduate, or unclassified); and major (science, humanities, or undeclared). Given 
conect information about the proportions of these characteristics in the popula- 
tion of ESL students at UHM, the researcher could tlien randomly select from 
each of die strata in proportion to those population characteristics. The sample 
that results would intentionally take on the same proportional characteristics 
found in the entire population. Creating a stratified random sample still requires 
random sampling, but has the advantage of providing a certain degree of preci- 
sion to die representativeness of die resulting sample — a fact wliich facilitates 
die use of die identified diaracteristics as variables in the study. 

Decisions about which strategy (random or stratified random) to employ in a 
particular study must be reached rationally, and in advance. Tliere are several 
considerations that must be kept in mind in making such decisions. First, it is 
generally useful to employ stratified random sampling when the population in 
question is fairly heterogeneous in nature. Tlie concern is that random sampling 
might not provide for selection from each of the strata, or subgroups, in die 
population. Second, a sti*atified random sample becomes imperative when the 
samples involved will be small or the groupings within the study will be un- 
equal in size. Third, it must be remembered that, if properly conducted, strati- 
fied random sampling has die advantage of letting the characteristics of the 
population determine which strata will be sampled. Hence, the stratified strat- 
egy is useful if die study will focus on the groups’ characteristics as moderator 
or control variables (see Brown, 1988, pp. 11-18). 

Alternatively, if die samples involved will be fairly large, straightforward ran- 
dom sampling can be employed. Random sampling is much easier to perform 
since diere is no need to define the characteristics of die population. It is only 
necessary to assume that the sample represents die population from which it 
was taken. Tliis assumption is widely accepted in research circles even diough 
it is counter-intuitive for some language teaching professionals. 

Sample Size 

One of the first questions diat will arise with regard to sampling is: How big 
must a sample be to be considered large enough? There is no easy answer to 
this question. However, it is clearly true that a large sample is better (in die 
sense of “more representative”) than a small one. Consider a sample whidi 
includes all but 1% of a population of 1,000 language students (that is, a sample 
that contains 99% of the population). It is likely that such a sample is more 
representative of the population dian one containing only 1% of it or 10% or 
30%. However, knowing tliis does not answer the question of how big a sample 
must be to be considered large. Unfortunately, sample size decisions depend on 
die situation involved in die study as well as upon the types of statistics diat will 
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be used. Statistics teachers will often give rules of thumb like the sample size 
should be at least 28 (or 30) per group or per variable. Tliis is not bad advice per 
se, however, such ailes of tluimb are usually vague and imprecise, and in any 
case are conveying the minimum number tliat you will need for correctly apply- 
ing many of the statistics tliat come up in research. 

Anotlier point of view is that, instead of estimating the bare minimum num- 
ber of subjects, the researcher should be estimating die minimum number of 
subjects that would be necessary for a statistically significant result to be ob- 
tained (if it really exists in the population) given the application of a particular 
statistical procedure under the conditions of die study that is being planned. 
Such estimadons can be made by using power analysis. One thing that power 
analysis can be used for is to analyze the relationship between the probability 
of finding a statistically significant result and the sample size given a particular 
set of expected results. If, for instance, a researcher wanted to esdmate the 
number of subjects that would be necessary to find a statistically significant 
difference between the means of two groups of subjects, it could be done 
mathematically on die basis of pilot data, or odier previous research that may be 
available in the literature. Such estimates can be made for a variety of the 
statistical procedures used for mean comparisons, correlation, and regression, 
as well as comparisons of frequencies (for more on power analysis, see Cohen, 
1988; Kraemer & Thiemann, 1987; Lipsey, 1990). Unfortunately, power analysis 
is mathematically complex. However, there is computer software available (e.g., 
Borenstein & Cohen, 1988) diat can resolve this problem. 

In short, when diinking about sample size, die best strategy is to make sure diat 
die population is clearly defined, and that the sampling procedures make sense. If 
pilot data or odier previous research is available, it wiU prove helpful to use power 
analysis to estimate die sample size diat is necessary to find a significant effea if it 
exists. If pilot data are not available, you may have to design your study such that 
die samples involved “seem” large enough to be representative, while keeping in 
mind diat a good aile of thumb is the larger the sample size the better. Tlie issues 
involved in sampling are somewhat subjective, and must in part be left up to die 
researclier. Sampling procedures are important pardy because of die way diat they 
affect die generalizability of the study. 

Tlie generalizability of a study can be defined as die degree to which die results 
are meaningful beyond die study itself widi regard to die entire population in 
question. If die sampling tecliniques have been properly conducted and the sample 
is large enough, diere should be no question in die researclier’s mind (or in a 
reader’s mind) as to the degree to which the sample represents die population. If 
diere is some question, dien die sampling techniques should be improved or the 
sample sizes increased, or both. (For more information on sampling and its use in 
language studies see Brown, 1988; Hatch & Lazaraton, 1991.) 
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Different Types of Variables 

A vanable is anything tliat can vary in a study. However, research is largely the 
study of what happens when variables are systematically manipulated in planned 
combinations. There are essentially five roles that variables can play in a study: 
dependent variables, independent variables, moderator variables, control vari- 
ables, and intervening variables. 

Tlie dependent vanable in a study is tlie variable of primary focus. It can also 
be thought of as tlie variable that is measured and studied to determine if other 
variables have an effect on it, or are related to it. The independent variable in a 
study is tlie variable tliat has been selected by the researcher in order to study 
its effect on tlie dependent variable (hence, the independent variable is some- 
times also called the manipulated variable). For instance, for a research ques- 
tion like “What is tlie effect of X on Y?”, X is die independent variable and Y is 
die dependent variable. Or, for a research question like “How well does X 
predict Y?”, X is die independent variable and Y is die dependent variable. 

Tlie relationsliip between die independent and dependent variables is central 
to any study. However, sometimes die researcher will also want to include a mod- 
erator variable in order to determine the effect of the moderator variable on the 
relationsliip between die dependent and independent variables. Thus, if a mod- 
erator variable were included, a question like the following could be posed: “What 
is die effect of X on Y when Z is present or absent^” In this last case, X is the 
independent variable, Y is the dependent variable, and Z is a moderator variable. 

In language research, there are usually variables other than the dependent, 
independent, and moderator variables which cannot be included in the design .or 
otherwise direcdy studied. Nonedieless, these variables must be accounted for 
often as control variables. Control variables are variables which are eliminated 
from die study, held constant, or odierwise kept from interfering with the study of 
die central relationship between the independent and dependent variables. For 
instance, in a study of the effect of Method A on English language proficiency (as 
measured by TOEFL), die researcher miglit compare the TOEFL scores of two 
groups, one who liad been taught by Method A and another who had received no 
instRiction, a control group. Tlie researcher would be most interested in the rela- 
tionsliip between die independent variable, Method, and die dependent variable, 
English Language Proficiency. However, there are a number of variables which 
might interfere widi die relationsliip between Method and English Language Pro- 
ficiency: gender, intelligence, aptitude, years of language study, etc. The researcher 
might choose to contiol gender by eliminating all males from the study. The re- 
searcher might furdier choose to use only students who had studied six years of 
English to hold die years-of-study variable constant. Random selection can also be 
used to create groups diat are dieoretically equal on all variables except those 
being manipulated as independent, dependent, and moderator variables. 
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Perhaps, die most confusion is caused by the term “intervening” variable 
because it is used in two distincdy different ways. On the one hand, intervening 
variable is used to describe the construct which underlies the relationship be- 
tween the independent and dependent variables. For instance, in die example 
study above on the effect of Method A on English Language Proficiency, die 
researcher might label die construct underlying die effect as “method effect” or 
“learning” or “language acquisition” depending on how it is conceptualized. 

On die other hand, intervening variable is used to describe a variable diat is 
unanticipated in a study, yet surfaces as being a possible explanation for the 
relationship between die independent and dependent variable. In the example 
study, it might turn out that any difference discovered in the proficiency scores 
of die Method A and Method B groups were caused by an unanticipated inter- 
vening variable rather than by the methods themselves. For instance, it might 
turn out that the teacher of die Method A class was just a better teacher than the 
teacher of the control group. Thus, a teacher effect turns out to be a possible 
intervening variable in the sense that it was unanticipated yet has potential 
explanatory power. 



Research Designs 

To understand die basic designs that are used in quantitative language studies, 
it will first be necessary to define some of the fundamental terms diat are used. 
The first idea that must be understood is that of a treatment. A treatment is 
something that the experimenter does to one group so as to study die effects of 
the treatment on die people involved. A treatment may be a specific teaching 
strategy, application of a set of materials, use of a particular reward system, or 
any other experience that the researcher wants to apply to the subjects for the 
study. Typically, for die sake of comparison, one group receives the treatment 
while another group does not. Thus, the subjects are divided into two or more 
groups: a control group and one or more experimental groups. The control 
group usually receives no treatment, or a placebo (some substitute that is pre- 
dicted to liave no effect), while the experimental group receives the treatment. 
In a language program, die treatment is likely to be some aspect of the language 
teacliing or learning experience. 

The reason for administering a treatment to the experimental group and 
nothing to die control group is to determine whether the treatment has had an 
effect. In order to do so, one or more observations must occur which allow for 
comparisons of the two types of groups. These observations may take many 
forms. In quantitative studies, observations may be simple tallies, rankings, or 
test scores. The point in making observations is diat something of interest to die 
researcher must be observed or measured so that comparisons can be made 
between the control and experimental groups. Naturally, whatever is observed 
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or measured must be related to die treatment. Tlius, in a language program, if 
the treatment was some form of pedagogy, you might be interested in observ- 
ing die language achievement test scores in order to determine the effect of die 
treatment on achievement. 

It is important to note diat studies involving anything other than examiriadon 
and description of test scores are difficult to conduct. The study may be de- 
signed in an airtight manner (difficult in any teacliing or learning situation), but 
in addition, considerable knowledge of statistics must be applied — usually more 
than die knowledge provided in one or two statistics courses. This warning is 
meant to encourage budding researchers to seek adequate guidance in design- 
ing quantitative studies and analyzing die statistical results. 

The two sections that follow will explain two categories of quantitative srtid- 
ies: tRie expeiimental designs and quasi-experimental ones. This is a very use- 
ful distinction explained much more fully in Campbell and Stanley ( 1963 ). 

True experimental designs are die most controlled language studies. They must 
be carefully planned from beginning to end. Hence, they are die closest diing in 
language studies to what most teacliers believe scientific experiments are like. One 
of die keys to identifying a true experimental design is that die subjects in the study 
must be randomly selected from the population being studied, and randomly 
assigned to die treatment or control group. Randomly is used here stricdy in the 
sense diat it was defined above. As described in die earlier section on Sampling, 
diis must be done so diat every member of the population has an equal chance of 
being selected. If diese procedures are followed and the resulting groups are laige 
enough, die researcher is justified in assuming that the two groups have very mucli 
die same cliaracteristics. Thus, true experimental designs have random selection as 
a precondition. The same tiling is true for posttest only designs, pretest-posttest 
designs, or any combination of die two. 

The posttest-only design (one type of true experimental design) is particularly 
dependent on random selection because it is assumed on the basis of sampling 
dieory diat the experimental and control groups are equivalent at the outset of 
the study. Such a study is designed as shown in Figure 1. Notice that step one is 
to use random selection to create equivalent groups. The experimental group 
receives the treaunent, while the control group does not (or receives a pla- 
cebo). Bodi groups are dien observed on the same scale and the performances 
of the two groups are compared. If die experimental group has significandy 
higher performance dian die control group, arguments can then be built that 
the treatment has an effect. The degree to which such claims can be made will 
nartirally depend on die magnitude of the differences in performance. 

The pretestposttest design, while it also assumes random selection of die two 
groups, allows die researdier to clieck die equivalence of the two groups at the 
beginning of die study, usually a pretest of some sort. Such a study would typically 
be laid out as shown in Figure 2. Hiis additional step allows for checking the 
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Figure 1 

Taie Experimental Design, Posttest Only 
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equivalence of tlie two groups in Step 2, but also allows for studying the amount 
of gain tliat has been made by each group between Steps 2 and 4. Tliis potential 
for sai dying gain allows the researcher to consider additional issues. For example, 
if tliere is a difference between the two groups on the posttest, die researcher can 
study whether the difference is as large as the difference between die pretest and 
posttest performances of die experimental group. If this is not true, the observed 
differences may have some source other than, or additional to, die treatment. 
Tlius, die pretest-posttest design is generally more powerful than the posttest only 
design because more inferences can be drawn. Pretest-posttest designs can be- 
come much more complex including various types of treatments used simulta- 
neously and various observation techniques used in the same study. 



Figure 2 

Taie Experimental Design, Pretest and Posttest 
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From a practical point of view, taie experimental designs are often doomed 
in real language teaching settings. First, students are rarely randomly selected. 
Thus, many researchers are working eitlier with what is called an intact group 
or with the entire population of students when they set out to do a study. 
Second, the researcher cannot set aside half of the students, randomly selected 
or othei*wise, to receive no language training, or a placebo. Either students want 
the training or they do not, and language researchers are seldom in \he moral or 
monetaiy position to simply withhold treatment (training) from one half while 
the other half receives training. As a result, language researchers are more likely 
to turn to what is called a quasi-experimental design. Quasi-experimental de- 
sigm, though less than perfectly controlled, provide useful alternatives to taie 
experimental designs. Quasi-experimental designs are adequate for die pur- 
poses of studying many language issues — particularly if no sweeping claims are 
going to be generalized from the results. According to Campbell and Stanley 
(1963), the main characteristic that makes a quasi-experimental design more 
practical for language studies is that the researcher has more control over the 
collection of data in terms of scheduling and deciding who will participate. 
However, the designs are weaker, and the results must be interpreted very 
carefully. Three types of quasi-experimental designs will be presented here: 
pretest-posttest designs (without control group), time series designs (without 
control group), and nonequivalent groups designs. 

The pretest-posttest design without control group is like the pretest-posttest 
design discussed above except that it lacks a control group. Such a design is 
shown in Figure 3- This type of design could be used as follows: A general 
proficiency pretest could be given at the beginning of a language program (the 
treatment) and again as a posttest at tlie end of die program. If diere is a large 
gain in average scores between the beginning and end of the program, it might 
be judged as a success. However, because there is no control group in such a 
study, the researcher can never know for sure diat the gain was not a result of 
language exposure outside of die program, or a result of a testing effect (that is, 
the effect of having taken the test twice), or a result of some other undeter- 
mined factor. In other words, the observed gains in scores may have been due 
to factors other than the learning that took place in the program. 



Figure 3 

Quasi-Experimental Design, Pretest and Posttest 
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Time series designs are more elaborate versions of the pretest-posttest design. 
The only stril<ing difference is tliat, in lieu of one pretest and one posttest, a series 
of observations, or tests, are made. Then, a treatment is inserted in the middle of 
tl-iis series. Such a design is described in Figure 4 (in wliich “O” stands for Obser- 
vation). In a time series design, the researcher can claim tliat tlie potential conse- 
quences of tlie testing effect mentioned above are controlled in tliat all students 
are made tliorouglily familiar with tlie format and content types on the observation 
instalments long before tlie treaunent comes into tlie picture. One problem that 
arises witli tliis type of design is tliat it sometimes calls for tlie development of 
numerous instalments, all of wliich must be very similar in what tliey measure. 



Figure 4 

Quasi-Experimental Design, Time Series 
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The nonequivalent groups design is different from the taie experimental pre- 
test-posttest design only in tliat the subjects are not randomly selected into tlie 
experimental and control groups. Such a design is shown in Figure 5. Because tlie 
groups are not randomly selected, tliey cannot be assumed to be equivalent at tlie 
beginning of tlie saidy. As a result, the equivalence of tlie groups must be checked 
in Step 2 (or otlieiwise controlled statistically). If it is possible to set up a control 
group in tliis manner and tlie groups do indeed prove to be equivalent at tlie 
beginning of tlie study, the nonequivalent groups design can prove fairly power- 
ful. However, if such a control group cannot be established, tlie quasi-experimen- 
tal version of tlie pretest-posttest design (Figure 3) may be the most effective 
design tliat can be used. 



Figure 5 

Quasi-Experimental Design, Nonequivalent Groups 
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There are many other types of complex designs (see Campbell & Stanley, 
1963, or Tuckman, 1978), and numerous ways of grouping and analyzing the 
results of those designs (see for instance, Keppel, 1973; Kirk, 1968; Pedhazur, 
1982; Tabachnick & Fidell, 1989). 



Validity 

The validity of a study can be defined as the degree to which the results can be 
accurately interpreted and effectively generalized. Tlie first part of this defini- 
tion — the degree to which die results can be accurately interpreted — is often 
referred to as internal validity. The second part, die degree to which the results 
can be generalized, is often labeled external validity. Table 1 lists the different 
factors that can affect the validity of a study (after Campbell & Stanley, 1963). 

Internal Validity 

The eight threats to internal validity, listed above, are variables that must be 
controlled in designing a smdy so that the results can be accurately interpreted. 

Histoiy includes anything that happens to the subjects, other than die in- 
tended treatment, between the observations in a study. For example, for the 
design shown in Table 1, history would be anything, other dian the treatment. 



Table 1 

Factors Threatening Validity (after Cambell & Stanley, 1963) 



Type of Validity 
Factor 

Internal Validity 

1. History 

2. Maturadon 

3. Testing 

4. InstRimentation 

5. Statistical regression 

6. Selection bias 

7. Experimental mortality 

8. Selection-niaturadon interacdon 

External Validity 

9. Reactive effects of tesdng 

10. Interaction of selecdon biases and the treatment 

11. Reactive effects of experimental arrangements 

12. Muldple treatment interference 
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that occurs between the pretest and posttest for eitlier the experimental or 
control group in the Taie Experimental Design, Pretest, and Posttest. 

Maturation refers to any of tlie processes in tlie subjects’ lives tliat occur be- 
cause of the passage of time and might interfere witli interpretation of tlie results of 
a smdy. For instance, fatigue, hunger, aging, changing schools, or passage tlirough 
pubeity would all be maairation factors tliat the researcher might want to consider. 

Test mg describes any influence that taking one test has on the scores of 
another test. For instance, taking the pretest shown in Table 1 might affect the 
scores on the posttest. The testing effect might be particularly pronounced if tlie 
type of test involved were completely new to tlie subjects involved. Consider a 
group of subjects who had never taken a cloze test before. If one were admin- 
istered 2 is a pretest, the subjects might learn test taking strategies that would 
make them more comfortable and make them score higher on a subsequent 
posttest, regardless of any treatment that was administered, 

Instmmentation involves the impact of variations in the tools of measure- 
ment or problems with the reliability of those tools (for much more on this latter 
topic, see Brown, 1995a, 1995b) on the obtained measurements, or scores. For 
example, a problem of instrumentation would arise if version A of a test was 
used in the pretest, but version B was used on the posttest. The problem is that 
any differences in performance could be due to discrepancies in the versions of 
the test (the instamients) rather than to any treatment involved. 

Statistical regression describes the moderating effects of selecting groups 
with extreme scores, eitlier very high, very low, or both. Under such conditions, 
the probability is that students with high scores will tend to score lower (i,e., 
closer to the average score), while students with very low scores will tend to 
score higher (i.e,, closer to the average score) for reasons having nothing to do 
with any treatments involved. 

Selection bias describes the impact of selecting subjects into the groups of a 
study for reasons other tlian chance. For instance, if the subjects for the experi- 
mental group in Table 1 were selected from students in 8:00 a.m, ESL classes, 
while the subjects in the control group were selected from students in 4:00 p.m. 
ESL classes, there might be differences in the groups based on class time prefer- 
ence that have nothing to do with the treatment involved. 

Experimental moiiality refers to tlie influence of subjects dropping out of one 
or more of the groups in a study. For example, in Table 1, some subjects in tlie 
conti'ol might be present for the pretest but absent for tlie posttest, Tliese absences 
might cause differences in tlie results that had notliing to do witli tlie treauiient. 

Selection-maturation interaction describes the effect of the maturation and 
selection bias variables (defined above) acting together. 

External Validity 

The four threats to external validity listed above can affect die generalizability 
of the results. 
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Reactive effects of testing and treatment describe die influence of taking a 
pretest on the sensitivity of the subjects to die treatment. For instance, if die 
treatment involved die use of cloze tests to practice reading prediction and the 
pretest was also a cloze test, the pretest might affect the subjects’ sensitivity to 
the treatment. In other words, the generalizability of the results might be -in 
question because the results depend on die use of a particular test. 

Interaction of selection biases and the treatment. If diere is some relationsliip 
between die group from which the subjects were selected and die effects of the 
treatment, interactions are said to exist between selection biases and the treat- 
nient variable. In other words, in any study, there is the possibility that any 
effects that are found are only taie for die population from wliich the groups 
were selected. It is also possible that the characteristics of diat particular popu- 
lation may cause the treatment to be effective where it would not be in another 
population. In such a situation the selection bias would be interacting with the 
treatment and thus affecting the generalizability of the results. 

Reactive effects of experimental arrangements. Tliis refers to die impact of die 
fact diat die a eatment was applied under experimental conditions rather dian real 
world conditions. For example, some pedagogical tecliniques might appear to 
work veiy well as a aeatment under classroom conditions but have no corre- 
spondingly beneficial effect on die students’ use of the language in the real world. 
The genemlizability of die results to the real world would be in question. 

Multiple treatment interference refers to die effects of applying more than one 
treatiiient to die same subjects. Under diese conditions, die effects of one treat- 
ment cannot be disentangled from die effects of others, and thus die results cannot 
be generalized to situations diat do not contain the multiple treatments. 

Multiple Threats to Validity 

Unfortunately, threats to validity in research are seldom as simple as Table 1 
would suggest. Tliis is because there may be numerous direats to validity oper- 
ating at the same time. Since the overall confusion caused by simultaneously 
having multiple direats to the validity of a study may well be synergistic, it is 
crucial diat researchers guard against and control any and all of these problem 
factors, preferably while planning a stiidy. (For more information on factors that 
threaten the validity of a stiidy and how to control diem, see Brown, 1988; 
Campbell & Stanley, 1963; Hatch & Lazaraton, 1991; Tuckman, 1978.) 



Ethics 

Ethics in social science research have been considered from a number of per- 
spectives. For an ovemew of this work see Kinimel (1988). Over the years, 
various organizations associated widi social sciences research have provided 
guidelines for their memberships. For example, die American Psychological 
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Association has provided various kinds of guidelines for the etliical conduct of 
research (American Psychological Association, 1953, 1981, 1982, & 1985). Ac- 
cording to Kimmel (1988), ethical problems in social sciences research may 
have a number of ihe following characteristics: 

1. Tlie complexity of a single research problem can give rise to multiple ques- 
tions of proper behavior. 

2. Sensitivity to etliical issues is necessary but not sufficient for solving them. 

3. Etliical problems are the results of conflicting values. 

4. Ethical problems can relate to both the subject matter of the research and the 
conduct of tlie research. 

5. An adequate understanding of an etliical problem sometimes requires a broad 
perspective based on tlie consequences of research. 

6. Etliical problems involve botli personal and professional elements. 

7. Ethical problems can pertain to science (as a body of knowledge) and to 
research (conducted in such a way as to protect the rights of society and 
research participants. 

8. Judgments about proper conduct lie on a continuum ranging from tlie clearly 
unethical to tlie clearly ethical. 

9. An etliical problem can be encountered as a result of a decision to conduct 
a particular study or a decision not to conduct the study. 

Commandments 

In language related research, some of the most important etliical and profes- 
sional issues might best be summed up by ten straightforward commandments 
(adapted from Brown 1984). Tliese commandments cover the researcher’s ethical 
and professional responsibilities with regard to the participants, analyses, and 
audience of a saidy: 

Participants 

I. Tliou shalt not abuse thy subjects in any manner including abuses of their 
persons, time, or effort, and thou shalt obtain tliy subjects’ informed con- 
sent if required by tliy institution. 

II. Thou shalt not abuse thy colleagues by collecting data from their students 
witliout permission, or by using too much precious class time. 

III. Thou shalt reward tliy subjects’ and colleagues’ efforts at least by giving 
tliem feedback or information on what happened in tlie study. 

Analyses 

Tliou shalt guard against consciously or subconsciously modifying thy 
data so that the results support thy views and prejudices. 

Thou shalt select the appropriate statistical tests. 

Thou shalt check the assumptions that underlie all statistical tests. 



IV. 

V. 

VI. 
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Audience 

VIL Thou shall explain thy research clearly so tliat it can be understood by 
thy readers. 

VIIL Thou shalt organize thy report using conventional sections, headings, 
and other conventions (see American Psychological Association, 1994) so 
tliat thy readers can easily follow thy study. 

IX. Thou shalt interpret tliy results carefully guarding against the temptation 
to over-interpret, or generalize beyond tliat which thy results warrant. 

Above All Else 

X. Thou shalt continue to learn, read, and grow as a researcher so that thou 
can better serve tliy field. 

Stating \he etliical issues in tlie form of commandments may at first seem to be 
intended as tongue-in-cheek humor, but these are not to be taken lightly. Indeed, 
the entire enteiprise of research in language studies hinges on cooperation be- 
tween subjects, colleagues, researcliers, and readers.* Researchers should avoid 
contiibuting to tlie already abundant negative feelings about statistical researcli. 



Conclusion 

This paper began with a disaission of the issues involved in sampling a group, 
or groups, of subjects to be used in a study. Then, data collection instalments 
were examined in terms of the four scales of measurement that can be used. 
Next, a number of research designs were explored. Then, the factors which may 
jeopardize the internal and external validity of language studies were surveyed. 
Finally, the ethical issues involved in collecting data, conducting researcli, and 
reporting tlie results were covered. In short, a great many crucial issues have 
been covered in this paper — issues that must be tliought tlirough before con- 
ducting a study. A little effort spent in the planning stages of a study can save 
tlie enormous amount of energy necessary to recover if tilings begin to come 
unraveled after the study has begun. 
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Research, simply put, is the gatliering of information (data) tliat will help to 
answer a question about the world around us. It can be broken down into two 
categories, secondaiy and primary. Primary research “is derived from primary 
sources of information (e.g., a group of students who are learning a language), 
rather than from secondary sources (e.g., books about students who are learn- 
ing a language)” (Brown, 1988, p. 1). Research also “involves planned and 
systematic inquiry” (Seliger & Shohamy, 1989, p. 12), which differentiates it 
from common sense. 

This is tlie same explanation we see in all the research manuals, and it is not 
what most teachers want to hear. Many language teachers have an ingrained 
fear of the veiy concept of research. They may say, “Research is for people with 
PhDs in the field. I just want to teach my courses.” Others may say, “Research 
requires statistics, and I don’t want to have anytliing to do with statistics.” 
However, research, whether we are willing to acknowledge it or not, is 
present in all facets of our teaching experience. How many times have you tried 
a new way of teaching vocabulary? How did you evaluate the efficacy of the 
technique? Did you compare test scores? Did you ask a few questions of the 
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students to see how tliey liked it? Did you overiiear some students’ remarks 
while they were doing the lesson? Did you look at their faces and notice that 
their eyes were lighting up with discovery— or clouding over witli confusion? If 
you have done any of tlie above, you have started down the road toward doing 
research. Of course, you did not formalize your hypotheses, write up a pro- 
posal, and submit it for publication, but you did a form of research all the same. 

Have you ever had to teach a pronunciation class and wondered wliich 
sounds to teach the students? Well, in order to find out, you probably did a kind 
of needs analysis. In other words, you may have interviewed some of the stu- 
dents to get a general feel for what types of sounds give tliem major difficulties. 
You may have given them a quiz in which they were to mark tlie correct answer 
to a question about how to make a particular sound. You may have given them 
a listening test in which they were to determine wliich word in a minimal pair 
was spoken. Or, you may simply have paid careful attention to tlieir pronuncia- 
tion while doing anotlier lesson. No matter how you went about it, tlie fact is, 
tlie needs analysis you did is a form of research. 

What we intend to do in this paper is to take some of the mystery out of the 
word “research” by showing you some ways you can carry it out without radi- 
cally changing what you normally do. In other words, you can do a lot of useful 
and interesting studies without ever leaving your classroom. 

Our goal is that tliis paper will help you to realize that research is something 
you can do, because you have been on the brink of doing it all along. The 
paper is divided into sections, with each covering a particular methodology. 
The sections will address the questions; What is it? When do I do it? How do I 
set it up? How do I analyze the results? And, where do I go from here? 



Action Research 



What is it? 

Action research (AR) could be described as classroom research on a shoestring. It 
is a kind of research that is accessible to every teacher. Simply put, to do action 
researcli all you need is a classroom, a problem, and an idea or two on how to 
solve tlie problem. As a good teacher that is exactly what you do every day. But 
what sets AR apart is tliat it is done in a more systematic manner. Action research 
formalizes what we do unconsciously every time we step into the classroom. 

When do I do it? 

Let us say that you are having trouble witli a class; the students are not respond- 
ing to your weU-prepared lessons, and you know tlie lessons are good because 
all your otlier classes are doing well. Or, a colleague tells you about a new way 
of teaching an old lesson, and you decide to try it. Or you are using a new 
textbook this year, one that you had no say in choosing. Is the book any good? 




68 



Methods for the ‘'Research Challenged 



73 



And how can you make it better? These are some of die more obvious times 
when action research can be a useful tool. 

Whereas many types of research — most notably experimental research widi 
all of its statistical analyses — are usually carried out by experts, action research 
is essentially the teacher’s ball game. In cooperation with other teachers and 
with students — and even widi full-time researchers — a problem is identified 
and worked on. From this perspective, die best dme to do AR is any dine. That 
is what makes tliis type of research so attractive to the classroom teacher. Talk 
to your colleagues, decide what to teach, try it, evaluate it, revise it, try it again, 
evaluate it again: Hey, you have just done your first action research study. 

We would like to point out the usefulness of a video camera. Even though it 
is troublesome to bring a camera into the classroom, set it up, and get usable 
video footage, the positives definitely outweigh die negatives. You do not have 
to watch the tape and transcribe all the utterances and actions; simply look at it. 
You will discover a lot of interesting tilings you never thought of, because you 
will be looking at it from a different perspective. It is also easier to consult with 
your colleagues, because you can show them die relevant parts of die lesson as 
they happened, as opposed to trying to explain how things went. 

How do I set it up? 

One aspect of AR that is often ignored is the need for collaboration. It cannot 
happen when teachers jealously guard dieir secrets for fear that they will be 
stolen. Get together with your fellow teachers, outline your problem, brain- 
storm some solutions, choose die most likely one, and plan your saidy. 

Once you have identified your problem and potential solution, it is time to 
conduct your research. As a guideline for what you should look for when ob- 
serving your students, Knowles (1990, pp. 7-10) offers the following questions: 

• What did die students actually do? 

• What were they learning? 

• How wordiwhile was it? 

• What did I do? 

• What did I learn? 

• What do I intend to do now? 

• What would I recommend to my colleagues? 

In essence, diese questions replace the formal hypotheses and research ques- 
tions found in experimental researcli. However, that does not mean that action 
research can be sloppy. Thinking about these questions combined with careful 
observation of what happens in your classroom is essential for carrying out a 
successfid AR study. So, do not just pass over these questions when you do 
some AR — write them down along widi die answers. It is just like die essay 
outlines you have your students write — not much fun, but really useftd. 
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How do I analyze the results? 

As with other aspects of action research, evaluation is also quite informal. Ob- 
serve what happens during the lesson, share your observations with your col- 
leagues, and discuss how effective your solution was and how to refine it. If tlie 
problem has disappeared, great! If not, go back to the drawing board, and 
consider other possible solutions. 

Where do I go from here? 

Action research is a great starting point for teachers who are allergic to research. 
It does not take a lot of resources or extra time, just a little foretliought and 
collaboration with your fellow teachers. So, read the other papers in this vol- 
ume, talk to your colleagues, observe your classes, and you will be on your 
way. And if you get bitten by the research bug, read on to find out how to dive 
into the ocean of hard research head first. 



Survey Research 



What is it? 

A second type of research that teachers often do unconsciously is survey re- 
search. Have you ever asked students for their opinions about a lesson? Have 
you asked them what they feel they need to study, what their weak points are, 
or what they want to discuss next week? If so, you have done survey research. 
Many teachers may not consider it so, but if research is the gathering of data to 
answer specific questions, tlien surveys certainly fit the bill. 

Survey studies “focus on a group’s attitudes, opinions, and/or characteristics” 
(Brown, 1988, p. 3) and are often exploratory in naaire. The researcher’s aim is 
to gatlier as much information as possible on a question rather tlian to confirm 
an answer. There are two basic types of survey research. The first is the use of 
questionnaires, which are especially useful in gathering data from a large num- 
ber of people. The second type of survey is the interview, which is very useful 
in gatliering detailed information from a smaller number of subjects. It is often 
used as a follow up to a questionnaire, but can also stand on its own. 

The best way to decide if a survey study is right for you is to look at what 
you want to achieve; examine all the parameters, such as who you want to 
gather information from and how easily you want to be able to analyze the 
data once it is collected. 

When do I do it? 

Survey research is especially useful in collecting descriptive data. Suppose you 
want to collect data on the needs of your students, tlieir opinions on some 
aspect of tlie lessons or class, ideas for future lessons, personal information, or 
anything of interest to you tiiat may not be easy to derive from a test — then 
surveys are a likely candidate. 

^0 
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Questionnaires can be useful when you need to collect information from the 
students in your classes. As an example, consider a curriculum revision project 
undertaken at a vocational school. As part of the overall curriculum revision 
project, a needs analysis was developed in die form of a questionnaire, both for 
the students and for their future employers. The information from the question- 
naire was then used to begin revising the curriculum. Once again, die informa- 
tion collected did not directly reveal the best curriculum but did provide much 
useful data for making informed decisions. 

On the downside, if you do not have a capdve audience Oike an EFL class) the 
response rate may be low. If die percentage of people who respond is too low, die 
value of die saidy will be called into quesdon. For instance, imagine that you send 
a class evaluation quesdonnaire to your saidents after die class has finished for the 
year — rafter all, you do not want the saidents to tliink diat dieir responses are going 
to influence dieir final grades. After a month, you look at die responses diat you 
received, and find out that 94% of the respondents diought your class was great, 
94% said your teacliing style was wonderful, and 94% believed your textbook was 
perfect. Tlien you notice diat out of 150 students, only 17 responded, and l6 (or 
94%) of diose 17 were A students. Well, most likely you already know that die A 
students liked you. What you need to know is why die B, C, D, or F saidents did 
not respond to your lessons. An interview may be useful here because it allows 
you to focus on specific informadon. Tlie one student who did not agree widi die 
majority might be able to provide more valuable informadon than the odier l6 
students did. As such, it would behoove you to follow up die quesdonnaire with 
an inteiview, either by telephone or in person, widi the student who may not liave 
liked you, might not have gotten much out of die textbook, or could have ben- 
efited from a different style of teacliing. 

How do I set U up? 

Setdng up a sinvey is bodi easy and difficult. It is easy if you diink that all you have 
to do is write up a bunch of quesdons, copy diem, give diem to your subjects, 
collect the papers, and read diem. However, you need to consider die following: 
Wliat informadon do you want to gadier? How stnictured do you want id Do you 
want open-ended questions so die subjects can write dieir answers freely, or do 
you want to limit the range of responses to make analysis easier? It is die actual 
wridng of the questions diat makes suivey research difficult. 

In this vein. Ary, Jacobs, and Razavieh, (1990, p. 422-424) have offered the 
following guidelines to help constmct an effecdve questionnaire: 

• Construct the instalment in such a way that it reflects quality. [The quesdon- 
naire should be carefully written, proofread, and laid out in order to ensure 
a high response rate.] 

• Keep the questionnaire as brief as possible, so diat it requires a minimum of 
the respondents’ time. 
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• Make sure that the respondents have the information necessaiy to answer 
the questions. 

• Phrase questionnaire items so tliat they can be understood by every respon- 
dent. 

• Keep individual questionnaire items as short and simple as possible. 

• Phrase questionnaire items so as to elicit unambiguous answers. 

• Phrase questionnaire items so as to avoid bias that might predetermine a 
respondent’s answer. 

• Avoid questionnaire items that inight mislead because of unstated assump- 
tions. 

• Make sure that the alternatives to each questionnaire item are exliaustive, 
tliat is, express all tlie possible alternatives on the issue. 

• Avoid questions that inight elicit reactions of embarrassment, suspicion, or 
hostility in tlie respondent. 

• Avoid “double-barreled” questions that attempt to ask two questions in one. 

To illustrate what this all means, consider the following examples of good and 
bad questionnaire items (Nunan, 1992, p. 143-144): 

• Good: What do you think about the proposal that foreign languages should 
be compulsory in high school? 

• Bad: Do you think that the concept of learner centeredness is utopian and 
unrealistic? [This is a leading question wliich attempts to predetermine the 
respondent’s answer.] 

• Good: Rank the following from 1 to 4 in order of preference. “I like to learn 
best by studying”: 

witli the whole class 
in small groups 
in pairs 
independently 

• Bad: Would you prefer a short, non-award course (3, 4, or 5 sessions) with 
part-day release (e.g., Wednesday afternoons) and one evening per week 
attendance with financial reimbursement for travel, or a longer, non-award 
course (6, 7, or 8 sessions) with full-day release, or the whole course de- 
signed on part-time release without evening attendance? [Besides being ex- 
tremely complex and confusing, tliis question attempts to gather information 
on more than one topic.] 

Types of questions include open-ended (completion or fill-in questions where 
respondents write tlieir answers freely), fixed choice (similar to multiple-choice 
test items), checklists (“Check all that apply”), ranking (“Rank the following 
from 1 to 5”), or scaled items (“On a scale of one to a hundred, how would you 
rate your teacher’s choice of necktie?”). 
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The scaled items are often written in the form of a Likert scale. Everyone who 
reads a popular magazine has probably run into this type of scale. For example: 

How do you feel when a dentist is looking at you with drill in hand and 

gleam in eye? 

1 2 3 4 5 

uncomfortable ecstatic 

Likert scales are especially useful when you need information on the opin- 
ions or attitudes of the respondents on issues that can be dichotomized, that is, 
separated into two opposite attimdes. Some of the common dichotomies found 
in language research are veiy serious/slight, important/unimportant, like/dis- 
like, and agree/disagree (Brown, ms), not to mention the ever-popular uncom- 
fortable/ecstatic distinction, as illustrated above. 

Hoio do I analyze the results? 

Sui-vey data, in general, will result in descriptive analyses rather than statistical 
analysis. If your questions are open-ended, you could theoretically end up with 
an infinite number of different responses — assuming, of course, tliat you had an 
infinite number of subjects who responded. You would then use these responses 
to tease out any patterns of interest for further investigation, such as interviews 
for more detailed data or an experimental study. Fixed choice and Likert scale 
questions, on the other hand, would most likely be analyzed as percentages, as 
in this example: 

Does your teacher arrive in the classroom on time? 

1 - 75% 2 - 22% 3 - 3% 4 - 0% 5 - 0% 

almost never almost always 

Again, one of the purposes in calculating the percentages is to discern pat- 
terns which merit further investigation. In our example, it would certainly be 
expedient to the researcher — or at the very least, die department chair — to in- 
vestigate the situation with the late teadier. 

Aldiough diis is die most common use for die data from a survey study, it is also 
possible to constiiict questionnaires or staicture interviews to invesdgate die rela- 
donship between different variables, such as die teacher being late and die stu- 
dents’ overall lildng of die teadier. In the example, die students responded diat 
they liked die teacher very much, most likely because dieir lesson was first period 
and diey Imew they could get away widi being 30 minutes late for class! 

Where do I go from here? 

We suggest that you take a look at your courses and ask yourself if the students 
seem to like or dislike a certain aspect, for example a reading or a video that 
you use. Constiuct a quesdonnaire and evaluate die data. You may find that 
they rate it higlily, in which case you would want to introduce more of die 
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same. Or, you may find that they rate it very poorly, and then you have to 
decide if the benefits of the material or activity offset die negative impressions 
of the students. At least you will have some sort of a starting point to work from. 



Correlational Research 



What is it? 

According to Brown (1988), correlational studies "are designed to investigate 
the nature and strengths of functional relationships among die variables of 
interest to the researcher” (p. 126). Anodier definition was offered by Hatch 
and Farhady (1982, p. 192): “In correlational studies, researchers are interested 
in determining the degree of relationship between pairs of two or more vari* 
ables. ... In other words, correlational studies allow us to determine the extent 
to which scores on one test are associated with scores on another test.” Corre- 
lational studies are pretty easy to comprehend, but the calculation of the "cor- 
relation coefficient” may give some people, including us, fits. Just keep in 
mind, though, that the calculations are the reason we pay big bucks for com- 
puters — and even bigger bucks to someone to am them for us. 

Basically, what you want to do is see if two scores or measurements are 
related to each other and how strong the relationship is. For example, one may 
compare the scores of a vocabulary test with those of a spelling test to see if 
they are related. The characteristics, in this case vocabulary ability and spelling 
ability (as measured by the scores on the two tests), are well defined, and the 
scores certainly vaiy from student to student. Once the strength of the relation- 
ship is established, one could make predictions based on that particular rela- 
tionship. It should be noted, however, that this is only a prediction and is not 
to be taken as something chiseled in stone. The problem with correlational 
studies is that they may not take unknown variables into account, or the vari- 
ables may not be as well-defined as we imagine. Also, it bears remembering 
that a veiy strong correlation does not mean anything more than that there is a 
strong relationship — causation is not established through a correlational study. 

When do I do it? 

A correlational study can be set up abiiost any time, as long as you have two or 
more variables that you wish to investigate. In order to keep it simple, we will just 
discuss studies in wliich two variables are compared, but die idea is die same for 
more than two. Most commonly, you will be comparing your shidents’ scores on 
different tests, such as an in-house proficiency test and the TOEFL. Some questions 
diat might be answered by a correlational study are die folowing: 

• To what degree does short-term memory relate to language proficiency? 

• How well do the ratings on compositions by two different judges match? 
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• Do students who use more English during classroom activities score higher 
on tlie end-of-term test? 

Note tliat in correlational research you are dealing with variables that exist 
more or less naturally, not ones tliat you manipulate. When you start mariipu- 
iating variables, such as using pairwork with one class and TPR with another, 
you are moving into the realtn of the experimental. Let us save that discussion 
for a bit later. 

How do I set it up? 

First, get a bunch of scores on one test. Then, get a bunch of scores on another 
test. Then, run the scores through a computer! Let us say that one day you 
notice tliat some of the students in your three-hour class seem more active after 
tlie mid-class break. You do some action research (or spying, as it is sometimes 
called) and observe tlie students during the break. You discover that half of 
them are going down to tlie local beer machine to have some refreshments 
before coming back up to class. So you ask yourself if there may not be some 
kind of relationship between drinking beer and participation. (See Guiora et al., 
1972 and Guiora et ai., 1980 for the effects of alcohol and other chemical sub- 
stances on second language pronunciation ability.) 

Next, you need to collect data. In our hypothetical example this would entail 
getting a “beer score” of how much the students are drinking and a “participa- 
tion improvement score” showing how much more the students participate 
during tlie second half of the class.. 

How do I analyze the results? 

Once you get all the data together, you have two ways of looking at it: visually 
and statistically. If you just want to get a rough idea of what your relationship looks 

like, do it visually. Make a simple 
graph with the X-axis being the 
scores for one of the variables (e.g., 
the beer score), going from lowest 
to highest, left to right The Y-axis 
would be the other variable (par- 
ticipation improvement score), once 
again going from lowest to high- 
est, bottom to top. The results may 
look something like the graph on 
the left. 

According to this graph, there is 
some kind of pattern which has de- 
veloped. There is a fairly strong le- 
0 10 20 30 40 50 60 70 80 90 100 lationslup between the two scores. 
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The left to right upward angle indicates that there is a positive degree of relation- 
ship here, and tliough tlie dots do not form a perfect line, it is pretty close. This 
means that, on the whole, the students who drank beer during the break actually 
did participate more during the second half of the lesson. But remember, that does 
not show that tlie beer caused the increased participation. 

Let us assume, however, that the following graph resulted when the data 
were mapped out: 
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This graph also shows a fairly 
straight line pattern, but going 
from top to bottom. That shows 
a negative correlation, or an in- 
verse relationship. In other words, 
as the drinking of beer goes up, 
the number of times a person has 
participated voluntarily has gone 
down in comparison to the par- 
ticipation for the non-beer drink- 
ers in class. This means that one 
could predict that the more one 
drinks beer during the break, the 
less one would speak. Again, 
however, that does not prove that 
the beer was the cause of the de- 
creased participation. 



Now take a look at the graph in this figure: 
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That is a random plot re- 
flected in that graph! There is no 
pattern at all. There is little or 
no relationship between the two 
variables. In other words, those 
who drank beer may have par- 
ticipated as much as, more, or 
less than those who did not. 
Therefore, it is very difficult to 
reach any conclusion about the 
relationship between beer drink- 
ing and classroom participation. 

Calculating a correlation coef- 
ficient is slightly more compli- 
cated than using the eye-ball 
method, but it is much more en- 
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lightening as it can give you infonnation on the strength of the relationsliip. If you 
are using a computer, just choose “correlation” from the menu and you are done. 
But because working witli tlie statistical calculations is more complex, we do not 
have tlie space, time, wherewitlial, or ability to go into furtlier detail. Therefore we 
suggest tliat you move on to the appropriate chapters in this book, and take a look 
at die references listed below. 

Where do I go from here? 

As you can see, correlational research is quite a bit more involved than action 
research or using surveys. However, it is a very useful tool in analyzing relation- 
ships between different variables, and you should seriously consider making it 
a part of your research arsenal. 



Experimental Research 



What is it? 

Experimental research is die oldest and most difficult type of research, and that 
is why we are not going to spend a lot of time confusing you about it here. Up 
to now, we have shown you diat action research helps to solve everyday prob- 
lems in eveiyday classrooms; surveys help you to discern patterns of responses; 
and correlational research helps you to discover relationships. Experimental 
research is for diose times when “you have really got to know!” 

You may wonder if diree hours per week of listening will improve students’ 
scores, as compared to the one hour per week you are teaching now. One may 
also ask if die amount of reading a student does has much effect on the student’s 
ability to spell. Does die amount of time a student uses the Internet relate to 
the student’s reading ability? In all of these cases, we are trying to see if there 
is some kind of cause-and-effect relationship between two variables wliich are 
well defined and easily measurable. To assess that relationship, the only thing 
to do, then, is to set up an experiment that isolates the two variables, get some 
data, and see how the data compare. 

When do I do it? 

Although die research purist may cringe, the ideal time for the average class- 
room teacher to set up and mn an experimental study is . . . never. It is our 
belief that the true heavyweight research should be undertaken by trained 
professionals. Because of the difficulty and expense of doing a proper experi- 
mental study, most teachers would be much better served by action research 
or some sort of survey study. That is not to say that teachers are incapable of 
doing statistical research — ^we are just trying to be realists. 

Having said that, we must now state that there is never a better time to get 
involved in liard experimental research than today! Teachers need to know where 
die researdiers are coming from, and what better way to find out than by doing it 
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ourselves. One reason for the growing popularity of action research is that the 
teacher is a central player. His or her problems, concerns, or questions are being 
directly addressed. Experimental research can do the same, but only if the teacher 
gets involved. Indeed, some researchers have aig^ued for greater involvement in 
research by everyday teachers (see Nunan, 1991; Johnson & Chen, 1992). 

How do I set it up? 

For now, the best way for teachers to get a general understanding of experi- 
mental research is to extend our earlier imaginary example. Recall that in the 
correlational study, the distinguished professor investigated the relationship 
between students’ beer drinking during the break and participation after the 
break. Let us suppose tliat tlie professor found a strong positive correlation; in 
other words, the students who drank beer(s) during the break actually did 
participate more in the second half of the lecture. This led the professor to 
wonder if tlie beer really was responsible for tlie increased performance, or 
whether anotlier unknown factor was at work. 

First die professor randomly separated the class into two groups. One group, 
called the control group, would receive no special treatment. The second group 
became die experimental group, which would receive a measured amount of 
beer. The first half of die following lecture was held as normal, and a participa- 
tion score was calculated for each “subject.” Then, during the break, all of the 
students were herded into a different room where the “treatment” was given. 
The control group received nonalcoholic beer during the 15-minute break, but 
the experimental group received a predetermined amount of genuine beer. 
After die break, the class resumed as usual, and again the amount of participa- 
tion for each student was recorded. 

How do I analyze the results? 

As we mentioned above, the goal in a typical experimental study is to determine 
whether a group that receives a treatment, such as a new way of teaching (or a 
controlled amount of alcohol), performs differendy from a group that receives no 
special treatment. Unfortunately, an explanation of the statistics needed to analyze 
die data from an experimental study properly is beyond the scope of diis chapter. 

Where do I go from here? 

If you are tRily interested in experimental research, we urge you to pick up a 
couple of good books, such as the ones listed below or the ones mentioned 
elsewhere (see Griffee, this volume). Once you understand the concepts in- 
volved, get a grasp on the types of analyses necessary to do justice to your data, 
and come up widi clear and precise research questions and hypodieses, you 
will be on your way. The time it takes to reach that point may be more time 
than most teachers are willing to invest, however. 

One way to begin to get a grasp on experimental research is to have a look at 
Brown (1991) and Brown (1992). These articles try to help teachers understand 
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statistical research. Tliey are not “how to do” articles, but more along tlie lines of 
“how to read and understand.” Along witli Brown’s 1988 book, Understanding 
Research in Second Language Learning, they are essential reading for any teacher 
who has sliied away from reading or carrying out experimental research. 



Conclusion 

We have attempted to introduce several types of research that may be of interest 
to teachers who, until now, have proclaimed a severe allergy to research. Al- 
though we have made light of research in this paper, we do not mean to imply 
that it is something tliat should be taken lightly. Every professional teacher 
needs to, at the veiy least, make an effort to understand research that is relevant 
to his or her teaching situation. We would also like to suggest tliat every teacher 
would be well seived by trying out some type of research in his or her own 
classroom. Your goal does not have to be publication of a refereed article in a 
leading journal. Indeed your goal does not even have to be to write any article 
at all. But, all teachers should feel tlie need to improve themselves, and tliat is 
what classroom research is all about. 

So, next time you have some trouble in your class, or next time a lesson bombs, 
or you are faced witli students who do not seem to respond, or you hear about a 
great new lesson and wonder if it could work in your classes, stop wondering. Get 
out tliere and do sometliing about it. Research may be the answer. 
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Chapter 7 



Action Research: 
Something for Everyone 



Gregory S. Hadley 

Keiwa College 



For many language teachers, the word “research” often evokes images of a 
controlled and scholarly project that is only understood by those in the mystic 
realms of linguists and second language acquisition research. Lightbrown (1985, 
p. 184) noted how in recent years researchers had “increasingly arranged their 
own research meetings, apart from the teaching conventions where they are 
always asked by someone in the audience to relate their findings to teaching 
practice.” On the otlier hand, most teachers say they are either unable or unwill- 
ing to attempt their own research project, saying they lack the time, back- 
ground, or experience to undertake such a project. Most teachers do not realize 
the benefits that come from doing small-scale action research projects. If they 
did, many would take pains not to miss such opportunities, and to invest them- 
selves in tlie adventure of classroom-centered research. 

The purpose of tliis article is to answer the following questions: What is 
action research? What obstacles must one overcome before doing an action 
research project? How can a new or inexperienced teacher begin an action 
research project? Finally, why should classroom practitioners attempt to do ac- 
tion research? After reading this article, language teachers will be prepared to 
start their own action research project. 
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What is Action Research? 

During the past few years, attention has focused on one form of classroom- 
centered research (CCR) which is called Action Research (AR). LoCastro (1994, 
p. 5) defined AR as “one form of CCR which is seen as being small scale and 
situational . . . focused on a particular problem, to try to understand and per- 
haps solve some concrete problem in an individual teacher’s classroom.” She 
also suggested tliat AR should not to be done by outside researcliers, but by 
achial classroom teachers. For a classroom practitioner who has never done an 
AR project, gaining tlie advice and assistance of an outside researcher or vet- 
eran teacher is quite beneficial. However, the classroom teacher should do the 
bulk of tlie work in order not to disrupt the natural flow of the class. 

Kemmis and McTaggart (1988) also suggested tliat teachers should use AR with 
the goal of solving a specific problem in their classrooms. They defined AR as a 
process of planning, observation, and reflection on the part of the teacher with the 
goal of finding a solution to specific classroom problems. Other experts in the field 
(Cohen & Manion, 1985; Nunan, 1992) have written similar definitions. 

I define AR as a process designed to improve teaching and facilitate learning by 
indentifying a specific classroom problem, mrgeting causes througli systematic 
data collection (surveys, observation, interviews, etc.), and applying an effective 
solution to the problem as a result of the data being collected and interpreted. 

Action research can be carried out by language teachers who do not have 
any special training in psychometric research metliods or statistical analysis. It 
takes the pragmatic potential of research out of academic settings and puts it 
back into everyday classrooms. To use an analogy from medicine, action re- 
searchers are not specialists but rather general practitioners who diagnose and 
prescribe remedies for the everyday illnesses (low motivation, undefined learner 
goals, etc.) that language classes are prone to catch. 



Obstacles to Action Research 

Barriers to starting an AR project are: Vague research ideas, professional isola- 
tion, lack of available resources, and lack of time. These must be addressed 
before starting the research. The first step is to develop a clear research idea. 

Developing a clear research idea may be difficult at first. Griffee (1994, p. 
19) said that most research begins with a simple idea which might be noticing 
something or wondering why something is the case. Griffee added that an- 
other way to get a research idea is to pay attention to the areas of pain and 
frustration in your teaching. Problems are prime motivators of AR projects. As 
teachers, we sometimes need to “come up against a brick wall” in our class- 
rooms before we are able to slow down and reflect upon what is not working 
well. Ideas for AR usually come naturally from this process of introspection. 

C' O 
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If focusing on areas of difficulty or discomfort proves too daunting or meta- 
physical, teachers interested in AR should devote time to reading current teacher’s 
journals tliat deal with problems, solutions, and issues that otliers have encoun- 
tered in tlieir classrooms. Sometimes such reading can reveal a problem tliat has 
previously gone undefined. 

After identifying a problem in your classroom situation, write your researcdi 
idea as a question tliat will help in finding a solution for tlie problem. It must be 
possible to find an answer to your question. For example, “Why do students 
come to class?” may be a vague and difficult question to answer, but tlie ques- 
tion “What classroom activities interest my students?” might be possible to an- 
swer tlirough systematic observation, surveys, or other research tools. 

Seek out other teachers while forming your research question. I believe tliat 
teachers improve their craft when in productive dialog with otlier professionals. 
If possible, join a language teachers’ union or organization. In Japan, tlie most 
influential professional organization that language teachers can choose to join is 
the Japan Association for Language Teaching (JALT), but tliere are other organi- 
zations as well (JACET, Nippon Communication Gakkai, etc.). It is possible that 
a local chapter of a professional teachers’ group meets in your city. If you do 
not have access to a local chapter, then seek out experienced teachers. Every 
city or region has at least one or two local “gurus” who are invaluable in helping 
new teachers get started. Find tlieni. At all costs, break tlie isolation and start to 
network with otlier teachers. Share your ideas and pedagogic problems witli 
them. New teachers are often surprised to find tlieir problem is common to 
many in tlie profession. If this is tlie case in your situation, otlier teachers may 
ask to collaborate with you on your AR project. 

After forming die research question and making contact with odier colleagues, 
search for resources. It is vital to gain a better understanding of your research 
ideas. If you do not check die background of your subject properly, you may 
find out later diat you have only repeated another person’s work. Joining a 
teachers’ organization such as JALT usually means you receive a regular maga- 
zine or journal full of interesting and helpful articles. Going to a regional or 
nadonal language teacliers’ conference provides an opportunity to browse through 
publishers’ displays of resource materials. 

Some teachers may live near a prefectural or university library. However, as a 
word of warning, learning how to cooperate with a local library bureaucracy may 
require an investment of dine before it bears niucli fniit. Having a computer widi 
a modem is also a great tool for a literature search. Baskin (1994) listed contact 
names and addresses, current costs, and procedures involved in searching for 
journal ardcles and documents in Japanese universides and computer networks. 
Teachers wisliing to do research in Japan would do well to read his article. 

Time is probably the major concern of a teacher wishing to embark on an AR 
project. It is difficult to make the time for reading or networking. There are 
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always many immediate and pressing demands. However, the time spent on 
idea formation and reading for AR is an investment that produces lasting divi- 
dends. One’s teaching and understanding of the processes involved in learning 
acquisition are enlianced by tlie insights gained from a literature search. The 
relationships we build through networking also enrich our lives. The increased 
rapport we gain witli students who respond to our efforts can make a world of 
difference in die day-to-day grind of die school year. Not making die time for 
diis type of endeavor is a lost opportunity. 



Doing Your Own Action Research Project 

Can a teacher widi no previous experience undertake AR? The answer is an em- 
phatic “yes.” This section will outline an AR project I began widi absolutely no 
knowledge of AR and wliidi resulted in an encouraging turnaround in my classes. 

The following points are paraphrased from Nu nan’s (1992, p. 19) seven-step 
cycle for AR: 

1. Initiation The teacher notices a problem in class. 

2. Preliminaiy Investigation The teacher spends time observing die class 

and taking notes of their behavior. 

After observation, die teacher forms a ques- 
tion or hypothesis as to the cause of the prob- 
lem. 

The teacher tries several solutions to solve 
die problem. 

After some weeks, the teacher consciously 
observes or measures the class again to see if 
diere has been any improvement. 

The teacher shares liis findings with odiers. 

The teacher looks for odier methods to solve 
his original classroom problem. 

Although I was not aware of Nunan's cycle when I started my project, I fol- 
lowed as similar process. 

Initiation: Teacher Notices a Problem in Class 

My teclinical college English class was not responding. From the first day of 
class, the students were slumping over dieir desks. They rarely looked up. 
When I greeted them, none returned die greeting, but instead looked furtively 
at each other before returning to their contemplation of their desks. Some slept 
through the class despite being awakened several times. To the simple ques- 
tion, “What’s your name?”, I several times received the response, “No.” I was 



3. Hypothesis 

4. Intervention 

5. Evaluation 

6. Dissemination 

7. Follow-up 
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assured by the school management tliat the students were merely shy and 
would eventually come out of their shells. However, the situation continued 
for weeks and tlien months. When no amount of class preparation seemed to 
work, I knew I had a real problem on my hands. 

The first thing I decided to do was to begin networking. I joined my local 
chapter of JALT. It was there that I met experienced and concerned teachers 
who were willing to help me. Drawing on tlieir advice, I found local libraries 
and requested catalogs from publishers of educational materials. I read texts 
that provided me with a better understanding of second language teaching 
methodology and theories. It was my hope that research could somehow help 
me find some solutions to my problem. 

Preliminary Investigation 

I read an article in Voe Jjanguage Teacher (Kobayashi, Redecop, & Porter, 1992) 
describing an AR project wliich examined Japanese college students’ motivation to 
learn English in relation to university entrance exams. Tlie findings were interest- 
ing for me because one of tlie groups tliat was surveyed was from a technical 
college. The autliors found tliat wliile tlie interest in English-speaking cultures was 
liigh amongst tlie students surveyed, tlieir intrinsic motivation — by which I mean 
motivation that is not attached to external rewards or incentives (see Deci, 1975, p- 
23 for a more detailed discussion) — to learn English was mucli lower than their 
university counterpaits. They speculated tliat perhaps lower scores on college 
entrance exams had sometliing to do with lower intrinsic motivation but left. the 
question open for furtlier investigation. 

Hypothesis: The Teacher Forms a Question After Observation 
It occurred to me that my students, who had just recently graduated from high 
school, might have intrinsic niotivation problems similar to those at the techni- 
cal college students mentioned in the above article. I formulated the following 
question: Is tlie intrinsic motivation of Japanese technical college students to 
learn English lowered due to their lack of participation in or inability to pass 
university entrance examinations? 

After forming my question, I began to read as much literature as I could find 
on the subject of motivation and the Japanese educational system. I quickly 
obtained books through publishers which helped me get a general idea about 
my topic. I also took advantage of tlie English research library at the Northern 
Illinois State University’s satellite campus in Nakajo, Niigata. I took my time in 
reading and absorbed as much information as I could. 

I then made a sui*vey similar in focus to the one that Kobayashi et al. (1992) 
administered to tlieir students (see Appendix 1). I omitted questions wliich did 
not pertain to technical college students, and added other questions designed 
to ascertain tlie students’ intrinsic motivation to learn English. The survey was 
then translated into Japanese and distributed to 83 students. 
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Table 1 

Comparison of Hadley 19913 and Kobayashi et al. 1992. 

'‘NA” signifies questions not used in the survey. 



Results of Survey (Percentage of Responses) 
Hadley 1993- 



1. Abroad to speak English 

Yes; 20% No: 80% 

2. Interested in foreign cultures 

Yes; 94% No; 6% 

3. Interested in explaining Japanese cul- 
ture 

Yes: 53% No; 47% 

4. Have a foreign friend 

Yes; 27% No; 73% 

5. Interested in English in Junior High 

Yes; 75% No: 25% 

6. Interested in speaking to a foreigner 

Yes; 100% No: 0% 

7. Interested in English in High School 

Yes; 76% No; 24% 

8. Want to speak to foreigners outside 
school 

Yes; 27% No; 73% 

9. English is important 

Yes: 75% No; 25% 

10. English most enjoyable 

Kin; 2% Elem; 6% JH; 30% 

HS; 46% Pres; l6% 

11. Took university entrance examination 

Yes: 18% No; 82% 

12. Examination facilitated speaking 

Yes; 19% No; 81% 

13. Watch English movies 

Often; 20% Some; 75% Never: 5% 

14. Class that motivates more 

Serious; 19% Relaxed: 81% 

15. Regularly read other sources of English 

Yes; 17% No; 83% 



Results of Survey (Percentage of Responses) 
Kobayashi, Redecop, and Porter (1992). 

1. Abroad to speak English 

Yes: 4% No: 96% 

2. Interested in foreign cultures 

Yes; 83% No: 17% 

3. Interested in explaining Japanese cul- 

ture 

Yes; 52% No: 48% 

4. Have a foreign friend 

Yes; 19% No: 81% 

5. Interested in English in Junior High 

Yes: NA No: NA 

6 Interested in speaking to a foreigner 
Yes; 73% No: 27% 

7. Interested in English in High School 

Yes: 60% No: 40% 

8. Want to speak to foreigners outside 

school 

Yes: NA No; NA 

9. English is important 

Yes; 87% No: 13% 

10. English most enjoyable 

Kin: NA Elem: NA JH: NA 
HS; NA Pres; NA 

11. Took university entrance examination 

Yes: NA No: NA 

12. Examination facilitated speaking 

Yes: 15% No: 85% 

13. Watch English movies 

Often: 8% Some: 50% Never: 42% 

14. Class that motivates more 

Serious: 12% Relaxed: 88% 

15. Regularly read other sources of English 

Yes: NA No: NA 
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I tallied tlie results (see Table 1) and compared tliem to die percentages from 
die Kobayaslii et al. (1992) saidy. I was surprised to find diat my saidents were all 
(100%) interested in speaking to foreigners (question 6), and almost all (94%) were 
interested in foreign culaires (q. 2). Aldiougli die Kobayaslii et al. saidy did not 
include a question about speaking to nadve speakers outside die classroom (q. 8), 
I added such a question because 1 felt diat it would be a significant indicator of die 
students’ intrinsic motivation. I noted diat die same students (27%) who already 
liad foreign friends (q. 4) were die only ones (27%) interested in speaking widi a 
foreigner outside die school (q. 8). Tlie low percentage indicated in question 8 
was especially suiprising since 20% of students in my sample had been abroad for 
die express purpose of speaking English, as opposed to die 4% in die Kobayaslii et 
al. saidy. I was also shocked to find diat only 18% of my students had even 
attempted to take a university entrance examination (q. 11). 

The suivey results were analysed using die Survey Power 2.1 software pack- 
age O^pson, 1992). The responses to questions 5, 7, and 10 suggested diat 
junior high school and high school were crucial times in the development of 
intrinsic motivation for studying English. The liighest percentage (76%) of stu- 
dents said they were interested in English during their high school years (q. 7). 
Almost half (46%), felt diat high school was the time when English study was 
most enjoyable (q. 10). This seemed to contradict everydiing that I had heard 
about the state of English language teaching in Japanese high schools. 

Analysis of die results showed diat 74% of the students who did not take an 
examination (q. 11) were also not interested in speaking to foreigners in English 
outside die classroom (q. 8); and, that die students who did not take a university 
examination were also not likely to read any English materials when not in 
school (q. 15). The implications of diis correlation were not examined in detail, 
although, at the time my colleages and I suspected diat non-pardcipation in 
university entrance examinations may have influenced the students’ intrinsic 
motivation to study English. 

The survey suceeded in identifing certain times and conditions when our 
students were more interested in English, and in demonstrating that intrinsic 
motivation had been stronger before die students arrived at the technical col- 
lege. Using the data from this survey, we began to tliink about some practical 
ways to try to revive that motivation. 

Intervention: Offering Solutions 

I discussed die results of the survey widi die rest of the native English speaker 
(NS) staff at the school and noted the following: (a) a high percentage of 
students interested in speaking to foreigners within a “safe” environment such 
as the school or at school functions, (b) a strong student interest in English 
movies, and (c) a desire — amongst the students — for a more relaxed classroom 
setting. Taking advantage of my role as head teacher at the school, odier NS 
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teachers and I attempted strategies we hoped would raise the students’ intrin- 
sic motivation to learn English, 

More native English speakers were invited to our seasonal school parties. In 
the past, the parties had been a closed affair which looked sadly like a middle 
school dance paity, but students began to interact witli the native speakers and 
use die opportunity to speak English in a natural setting. Together with other NS 
teachers at the school, we introduced video to die classroom, using focused 
listening and information gap activities. We knew how important pop music 
was to our students, so we incorporated a number of acdvities using music (see 
Griffee, 1992). To foster a more relaxed classroom setting, we changed our old 
texts which were based on an audiolingual syllabus and switched to a more 
comniunicadve-based syllabus. We included many task-based classroom acdvi- 
des to get students out of dieir chairs and speaking to other group members. 
Borrowing ideas from Moskowitz (1978) as well as Davis and Rinvolucri (1990), 
we used more “caring and sharing” activities so that students could express 
themselves on issues important to them, such as dating, jobs, and entertain- 
ment. We encouraged them to work together in their language learning. 

On occasion, we took our students out of the classroom and taught in parks, 
markets, and coffee shops. Although not all the students warmed up to it, we 
instituted a drama day at the school where students could exercise dieir skills 
in theater and role play. 

Evaluation 

After three months, all teachers observed a higher level of classroom participa- 
tion. Whereas students had often been absent, unresponsive, or asleep, we 
found that the majority were laugliing, speaking, and even joking with the 
teachers in class. Several of the students made new friends or penpals with 
nadve English speakers outside the school, and the students began to have 
more conversations with teachers outside of class. After graduation, a number 
of students went on extended personal trips to Australia, England, and the 
United States for die purposes of ineedng penpals, learning more about En- 
glish-speaking cultures, and improving their language skills. We saw a dramadc 
turnaround in our students’ modvation thanks, in part, to our efforst to apply 
what we learned through AR. 

Dissemination: Sharing the Findings with Others 

Teachers can share the findings of their AR projects in local teachers’ meetings 
as a presentation, in informal meedngs with other colleagues, or by publishing 
their results for a larger body of readers. While many teachers seem to find 
articles written by linguists to be dry and unrelated to their classrooms, the same 
teachers are quick to listen to the findings of a colleague who shares the same 
day-to-day difficulties. Acdon research reports get read and appear to have 
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greater immediate impact on the practices of other classroom teachers tlian tlie 
findings of second language researchers. 



Follow-up: New Solutiom 

Out of curiosity, I networked with otlier teachers at technical colleges. I wanted 
to see if the sui-vey results from my school would be similar if we took a larger 
sampling from other schools in Niigata prefecture and the Tokyo metropolitan 
area. We administered the same sui*vey to 562 students. The results were indeed 
similar, yet not as clearly delineated as in the original sui*vey (see Appendix 2). 
This follow-up of my research on a larger scale lent credence to my earlier 
findings, and helped me to show my colleagues tliat the dynamics present in 
my school might also be present in theirs, hopehilly spurring them on to begin- 
ning AR projects of their own. . 

My conliact reached its two year limit, and I moved on to teach at tlie college 
level. If I had had more time at tlie teclinical college, I would have looked for 
solutions in addition to tlie ones we tried. Tliis is anotlier aspect of AR: The process 
is constantly in motion. One cannot afford to rest on one’s past accomplishments, 
but must move foiward to keep on tlie cutting edge of the class’s task of learning. 



Wliy Should Teachers Do Action Research? 

The advantages of having AR in one’s “teacliing toolbox” are clear. Apart from 
improving one’s craft, teachers will gain a greater insight into what is going on in 
tlie minds of tlieir suidents. Clearer communication will be fostered between teacher 
and student. Few teachers work well in a professional vacuum, so opportunities 
for networldng witli otlier dynamic and conscientious educators is a definite plus. 

Instead of being a reactive teacher who fumbles in the darkness, groping for 
sometliing tliat will work in class for tliat day, one can become a pro-active teacher 
tlirough tlie thoughthil use of AR. Based upon tlie data gained from research, a 
teacher can seize the inoment and move forward witli purpose and clarity. 



Conclusion 

When I arrived in Japan over tliree years ago as an inexperienced teaclier, I 
encountered various problems. Not only did I feel isolated and overworked, I 
also had to deal with culture shock. At the same time, I was struggling to find 
ways to reach a particularly gloomy, poorly-motivated group of teclinical col- 
lege students in a dilapidated building. Tlie administration had no training pro- 
gram for new teachers. Our teaching materials consisted of tattered 15-year-old 
TEST textbooks obtained from what appeared to be an American college bar- 
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gain basement sale. I was oveiwhelmed and didn’t know where to go for help. 
It is in this type of environment tliat AR flourishes. 



References 

Baskin, R. (1994). Research in Japan. The Language Teacher 18 14-15, 110. 

Cohen, L., & Manion, L. (1985) Research methods in education. London: Groom Helm. 
Davis, R, & Rinvolucri, R. (1990). The confidence book. Essex, UK: Longman. 

Deci, E. L. (1975). Intrinsic motivation. New York: Plenum Press. 

Griffee, D. (1994). The art of language research. The Language Teacher 18 (2), 19-21. 
Griffee, D. (1992). Songs in action. London: Prentice Hall. 

Jepson, T. (1993) Suwey Power 2.1 for DOS. Wisconsin Rapids, WI: Wisco Computing 
Company. 

Kemmis, S., & McTaggart, R. (Eds.). (1988). The action research planner (3rd ed.). 

Geelong, Australia: Deakin University Press. 

Kobayashi, S., Redecop, B., & Porter, R. (1992). Motivation of college English students. 
The Language Teacher 16 (1), 7-9, 15. 

Lightbrown, P. M. (1985). Great Expectations: Second Language Acquisition Research 
and Classroom Teaching. Applied Linguistics 6" 184. 

LoCastro, V. (1994). Teachers Helping Themselves: Classroom Research and Action Re- 
search. The Language Teacher 18 (2), 4-7. 

Moskowitz, G. (1978). Caring and sharing in the foreign classroom. Rowley, MA: Newbury 
House. 

Nunan, D. (1992). Research methods in language learning. Cambridge: CUP. 



Action Research 



97 



Appendix 1 
Survey 



1 . Have you been abroad to speak English? 

[ ] A. Yes 
[ ] B. No 

2. Ai'e you interested in foreign cultures? 

[ ] A. Yes 
[ ] B. No 

3. Are you interested in explaining Japa- 
nese culture to foreigners in English? 

[ ] A. Yes 
[ ] B. No 

4. Do you have any foreign friends? 

[ ] A. Yes 
[ ] B. No 

5. Were you interested in studying English 
in junior high school? 

[ ] A. Yes 
[ ] B. No 

6. Are you interested in speaking to a for- 
eigner? 

[ ] A. Yes 
[ ] B. No 

7. Were you interested in studying English 
in high school? 

[ ] A. Yes 
[ ] B. No 

8. Do you ever tiy to speak English with 
foreigners outside the school? 

[ ] A. Yes 
[ ] B. No 

9. Do you think that English is important 
for your future job? 

[ ] A. Yes 
[ ] B. No 



10. When did you enjoy English the most 
(circle one)? 

[ ] A. Kindergarten 
[ ] B. Elementary School 
[ ] C. Junior High School 
[ ] D. High School 
[ ] E. Present School 

11. Did you take a University Entrance Ex- 
amination? 

[ ] A. Yes 
[ ] B. No 

12. Did the university entrance examina- 
tions help you to speak English? 

[ ] A. Yes 
[ ] B. No 

13. How often do you watch English mov- 
ies or videos? 

[ ] A. Often 
[ ] B. Sometimes 
[ ] C. Never 

14. Which type of class motivates you more, 
a serious or relaxed class? 

[ ] A. Serious 
[ ] B. Relaxed 

15. Do you regularly read books, maga- 
zines or newspapers in English (not includ- 
ing your textbook)? 

[ ] A. Yes 
[ ] B. No 
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Appendix 2 

Motivation of English Technical College Students: Results of Sui-vey (Percentage 



of Responses) Hadley, Fountaine, and Megill 1993- 




1. 


Abroad to speak English 


Yes: 44% 


No: 56% 


2. 


Interested in foreign cultures 


Yes: 90% 


No: 10% 


3. 


Interested in explaining Japanese culture 


Yes: 57% 


No: 43% 


4. 


Have a foreign friend 


Yes: 38% 


No: 62% 


5. 


Interested in English in Junior High 


Yes: 68% 


No: 32% 


6. 


Interested in speaking to a foreigner 


Yes: 93% 


No: 7% 


7. 


Interested in English in High School 


Yes: 65% 


No: 35% 


8 . 


Want to speak with foreigner 


Yes: 37% 


No: 63% 


9. 


English is important 


Yes: 79% 


No: 21% 


10. 


English most enjoyable 


Kindergarten: 7% 






Elementary School: 8% 






Junior High: 21% 






High School: 45% 






Present School: 


23% 


11. 


Took university entrance examination 


Yes: 31% 


No: 69% 


12. 


Examination facilitated speaking 


Yes: 33% 


No: 67% 


13. 


Watch English movies 


Often: 29% 








Sometimes: 64% 








Never: 7% 




14. 


Class that motivates more 


Serious: 23% 


Relaxed: 


15. 


Regularly read other sources of English 


Yes: 20% 


No: 80% 
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Chapter 8 



Students as Ethnographers 



Linda Donan 

In a busy Osaka fast food restaurant at lunch time, three Japanese junior coUege 
saidents suneptitiously position, tliemselves to watdi a foreign couple eating. One 
student takes note of how often tlie woman uses her napkin compared to the man; 
anotlier student counts the times the two customers reach across the narrow table 
to touch each otlier during tlieir conversation; and tlie third student uses two 
stopwatdies, a pencil, and paper to record how much time tlie couple spend in 
verbal exchange and in silence. As soon as the couple finishes their lundi, the 
saidents select an entering Japanese couple who are apparently about the same 
age as the foreign couple, and set up to resume their surreptitious observation. 

Meanwhile, in Mie prefecaire, an American researcher has arrived at a small 
fishing village. He is being formally introduced to the mayor by a go-between 
who is also a researcher at a respected Japanese University. Later, the mayor 
will urge various village members to aUow the researcher to live for extended 
periods in their homes. After a year or more, the researcher will have suffi- 
ciently ensconced himself into tlie village life, to report on tlie modern “house- 
husband” role attributed to tlie partners of traditional ama (female pearl divers). 

Botli die professional researcher, dedicating a year or more of his life, and die 
college students giving up dieir lundi hour, are practising edinographic research. 



Ethnography 

The word edinography is formed from die two Greek words “ethnos” (tribe or 
nadon) and “graphos” (sonietliing written). Bodi the researcher and the college 
students are studying members of a tribe or nadon. Bodi the professional re- 
searcher with his suitcase full of notebooks, cassette tape recorders, word pro- 
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cessor, video, and time-lapse cameras; and die students with their simple timers, 
notebooks, and pencils are doing their ethnographic study by writing some- 
diing down or recording what diey observe. 

The edinographer, Clifford Geertz (1973) noted that it is die act of inscripdon . 
which transforms momentary events into accounts diat can be reconsulted. 

If anyone should ever wish to know who washes the dishes in the home of 
the ama, or if anyone wants a glimpse at die eating habits of foreigners in fast 
food restaurants in Japan, they can consult a written report. 

Miles and Huberman (1984) pointed to the use of edinography in providing 
rich descripdons of local communides. The pearl diving village described above 
is a local context chosen for its unusual characteristics. For the purpose of cross- 
culairal or comparadve research, however, a field such as the fast food restau- 
rant selected by die college students is richer because people of many cultures 
can be found performing the same ritual— eadng their hamburgers. 

Frey (1988) contrasted die subjecdvity of ethnography with die objecdvity of 
fact finding. Although diere are tools of measurement used in both of the ex- 
amples described above, die most important tools are the observers diemselves. 
The edinographic invesdgator looks at a subject that they are ideally equipped 
to understand, namely, human beings. Observing acdons, listening to discourses, 
and in longer studies discussing the subjects' perceptions of diemselves are the 
primary ways diat data is gathered for edinographic research. 

Ethnographers also differ from experimental researchers in dieir reladonship to 
the acdons they observe. Wliile the latter generally attempt to maintain a personal 
distance from the observadon, ethnographers often interact closely widi their sub- 
jects or with the subjects' environment. Lindlof (1988) extended this nodon of 
involvement to include ethnographic descripdons that Require the richness of the 
observers' experience in die field to give fullness to die situadon being described. 



Why Choose Ethnography? 

Experiments, or odier forms of quandtadve research, are useful when working 
with numerically measurable data under controlled conditions. They provide 
excellent techniques for getdng very specific information. They are not, how- 
ever, always appropriate for working with human subjects. People often act in 
complex and unexpected ways that are beyond die parameters of an experi- 
mental hypothesis. Experimental exacdtude, wliich might require controlled 
condidons, tends to make people behave self-consciously or unnaturally. 

Ethnographers and other qualitadve researchers try to get a wider view. They 
trade specificity and measurability for the experiential depth of watching what 
people do in dieir normal acdvities in their normal life spaces. 

Qualitadve and quandtadve research are often viewed as polar opposites 
appropriate for different tasks or different departmental paradigms (i.e., quand- 
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tative research for the hard sciences and qualitative research for the social sci- 
ences). It is possible tliat some types of exploration may best be carried out 
tlirough a series of research strategies beginning with ethnographic research 
and progressing tlirough to systematic quantitative studies. The choice to be an 
ethnographer rather tlian an experimental researcher can also involve issues of 
personal conviction based on personality traits, attitudes, and values. 



What are the Steps of Ethnography? 

Careful planning and procedures can help to insure a positive outcome to eth- 
nographic studies. Some steps you may want to consider are: 

1. Preplanning 

(a) Choosing a study place (field) or subject (tribe). 

(b) Determining limitations and possible problems. 

(c) Casing tlie scene for feasibility and suitability. 

(d) Making your formal research proposal. 

(e) Gathering tools and arranging logistics. 

2. Getting in and getting help 

(a) Protecting human subjects. 

(b) Convincing gatekeepers to allow access. 

(c) Making use of a “friendly native.” 

3. The actual observation 

(a) Choosing a role. 

(b) Taking field notes and processing them. 

(c) Interviewing tlie observed. 

4. Follow-up 

(a) Analyzing your notes. 

(b) Developing concepts. 

(c) Interpreting and evaluating. 

(d) Writing up and publishing results. 

The above list is not to be considered as an inflexible procedure, but rather as 
a set of useful choices. Each etlinographer must follow the steps tliat prove 
successful — tlirough trial and error — ^with their own subjects, in their own fields. 

1. Pre-planning considerations 

(a) Choosing a study place (field) or subject (tribe). Determining which cul- 
tures and wliich subjects to study is often based on personal interest and oppor- 
tunity. The Mie researcher described earlier in this paper, came from a society 
where shared household duties is a modern norm to study tliis marginal phe- 
nomena in a society where household roles are usually divided between the 
sexes. For tlie college students a sentence in an English textbook about the 
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fastidious eating habits of Japanese people prompted tlieir study. Their teacher 
had read earlier research results comparing touching customs in various cul- 
tures and had suggested tliat research question to them. 

Having decided on die subjects, one must then choose die field or place of 
obsei-vadon. For college students, unable to travel abroad, the local fast food res- 
taurant provided an accessible site for ethnographic research. However, if foreign 
restaurant-goers never entered diat restaurant then it obviously could not be cho- 
sen as a field of research. If lighting was low, tables were curtained off, or if the 
students could not get a table diey would have had observation difficulties. 

(b) Determining limitations and possible problems. The tribe and field one 
chooses to study will produce unique problems that must be considered. 

The researcher’s own limitations must also be considered. The Mie researcher 
had considered his own gender, his height, liis clothing, and obvious foreign- 
ness and taken into account how those factors may effect liis interaction with 
the subjects. He had prepared himself by studying the Japanese language, but 
had also taken into account his understanding of Japanese culture and potential 
ethnocentrism or prejudices tiiat might color his observations. Likewise, his 
own positive attitudes towards males doing housework needed to be seen as a 
possible problem tiiat might affect his findings. 

The college students’ inexperience at research was compensated for by tiie 
simple numeric techniques tiiey chose. Nevertheless, one of the students re- 
vealed the effects of etlinocentrism on observation when testing her hypothesis 
tiiat tiie Japanese subjects would use their napkins more often than the foreign 
subjects. She had equated napkin use with a personal value of cleanliness. Her 
results, however, contradicted her hypothesis, a fact which she found so per- 
sonally and emotionally upsetting that she re-did tiie observation several times. 
Her co-researchers had to ask her to be silent in order to prevent her incredu- 
lous and angry remarks being overheard by the subjects under observation. 

(c) Casing the scene for feasibility and suitability. The next step, casing the 
scene, does not necessarily start at tiie scene. If at all possible, ethnographers 
should consult tiie literature for previous or on-going studies of tlieir subjects, and 
consider tiie recommendations of experienced people before tiiey visit tiie field. A 
researcher studying African tribes was once given the unexpected advice, by an 
experienced etiinogmpher, to carry a stick to ward-off dog attacks (Sanjek, 1990). 

Colleagues of tiie Mie researdier liad told liim of the ama and their domestic 
an-angements, and he had read anecdotal reports as well as Japanese research 
studies. He also read about mainstream Japanese attitudes towards housework, 
and discussed tiie issues involved with his Japanese acquaintances. The college 
students had discovered tiiat only the one of them tiiat was sitting nearest tiie table 
under observation could hear well enough over the noise to be certain whether 
tiie subjects were speaking or not. They realised, however, that tiiey could some- 
times watch tiie subjects’ lips eitiier directly or reflected in a nearby wall mirror. 
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(d) Making your formal research proposal For many ethnographers, the next 
task involves setting up a formal research structure. Tlie Mie researcher had to 
submit a research proposal to get time off from liis university teacliing. He also had 
to convince die funding board of liis rationale for die study. The college students 
had received dieir teacher’s verbal approval of dieir project, and they had arranged 
a meeting time and decided who would do wliich part of die observation. 

(e) Gathering tools and airanging logistics. Equipment needs can also be deter- 
mined and prepared in advance. Tradidonally, an edinographer’s tools liave in- 
cluded two eyes, notebook, and a pencil. Recendy, recording and counting de- 
vices have aided observation, but one should be careful to take into the field only 
those tools diat can be handled easily wliile focusing attention on die subjects. 

If possible, the observer should conduct a pilot study or infomial visit to a 
proposed site to determine such things as die best time to conduct the observa- 
tion, and the best vantage point. Earlier studies in Mie by Japanese anthropolo- 
gists focusing on the ama had prepared die villagers for the inconveniences of 
tape recorders and strangers in their homes. For die student observers, die 
lunch hour aish proved an ideal time to observe subjects from a well centered 
table that afforded a view of all odier tables. 

2. Gettmg in and getting help 

(a) Protecting human subjects. Care must be taken diat the subjects of edino- 
graphic research be allowed to go about their natural lives widiout feeling 
imposed upon by the obseivation process. 

In America, where experimental studies in bygone decades allowed researchers 
to shock their subjects with electrodes, strict measures are now in place to 
protect human research subjects. The American researcher, therefore, had to fill 
out legal documents to be considered by a board in charge of the safety and 
happiness of human research subjects. Throughout his stay, he had to ask again 
and again if people minded him sitting in on dieir daily activities, and request 
permission for all his intaisions including die use of tape recorders and video 
cameras. Any slight hesitancy or protest on the part of his subjects required him 
to retract and wait until the people were more comfortable with his presence, or 
to withdraw to another home. 

No such standards were set for die students in Japan. Their teacher did 
suggestion that the content of private conversations not be recorded, and in 
another case from the same class the teacher suggested that the interviewed 
foreigner not be approached for a "date.” 

(b) Convincing gatekeepers to allow access. Getting into the research field 
often requires admission by a "gatekeeper.” The Mie researcher was introduced 
by a prominent professor affiliated with Tokyo University. The Mayor also worked 
on liis behalf securing die confidence of local people. Tliese go-betweens helped 
to smooth the way for the study. The students could have asked for permission 
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from the restaurant's employees and manager but chose to remain incognito — 
even to tlieir gatekeepers — and to rely on the fact tliat one of tliem was a former 
employee to get them out of any trouble if it arose. 

(c) Making use of a friendly native. ” Japanese people are well aware of die 
uses of go-betweens as gatekeepers and can accept the need for a “friendly native” 
to help widi ti-anslating not only die language of die subjects but also die culture. 
Especially in observing a culhire or sub-culhire odier dian one’s own, die help of 
a friendly nadve can save die researdier from errors such as that of die edinogra- 
pher, Margaret Meade (Howard, 1984). Meade did not find out until years later diat 
die subjects of her soudi sea island research had been “pulling her leg” with dieir 
sexual stories because she seemed to enjoy sucli tales. She did not have a friendly 
nadve who was frank enough widi her to explain diat her research was being 
skewed by die iniaginadve stories diat diey thought would please her. 

5. The Observation 

(a) Choosing a role. Ethnographic researchers must choose a role to play in 
the field. Some possible roles are: complete pardcipant, pardcipant-as-observer, 
observer-as-paiticipant, and complete observer. The college shidents pretend- 
ing to be customers and hiding their pencils behind their lunch were aiming at 
die complete participant role. Tliis role can be particularly effecdve in diat 
subjects are usually unaware of being observed. It is also effecdve in diat the 
researcher shares die experience of the subjects, and knows, for instance, if the 
greasiness of die fries makes one want to use a napkin or if the noise level of 
the restaurant discourages conversadon. 

The work of Trujillo and Dionisopoulos (1987) observing policemen’s use of 
“labeling” and “talking tough” by riding along with diem in die front seats of 
pati'ol cars is an example of pardcipant-as-observer. They were known to be 
conducting research and were not wearing police uniforms. They were not 
expected to take an acdve part in arrests, and could completely wididraw from 
dangerous siaiadons. They negodated a space to cohabit, and the police never 
expected them — nor would allow them — to “face the heat.” 

Pardcipant-as-observer was also die role chosen by the Mie researcher. He 
could not pass as a Japanese nor become a second house-husband in the situ- 
ation. Aldiough he hoped to be granted the comfort of being included in die 
family over dme, he remained aware of the limitations of liis identity as a guest, 
and maintained an awareness of his obtrusiveness. 

An example of the observer-as-pardcipant role is the interview. In tliis situa- 
don, the researcher is on die scene for a brief period and clearly playing the role 
of observer. Their clipboard is their badge that allows them to stop complete 
strangers on die street and take up dieir time with quesdons. Researchers who 
do not make dieir role clear, for instance during random street interviews, might 
receive liosdle reacdons, be regarded as crazy, or even be reported to police. 
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To be a complete obsei-ver, in otlier words to see and not take part or be 
seen, is possible these days witli the use of cameras and tape recorders. Many 
social scientists dislike tliis method of gathering data as it can stretch the rules of 
privacy and it does not allow die researcher enough contact widi his or her 
subject. Clearly, the categories are not simple or definite but rather there is a 
spectrum of possibilities. The Mie researcher may have pitched in and washed 
dishes with his house-husband hosts, and die students I am sure took some sips 
of their shakes and bites of their hamburgers. 

(b) Taking field notes and processing them. Taking field notes is die heart of 
ethnography. Questionnaires and taped interviews can be quite comfortable 
tools for researchers interacting in a culture and language that is not their own. 
The class of the Japanese college students in the above-mentioned class in 
Osaka had learned diat neither of diese tools were as effective as observation, 
for their paiticular questions. A questionnaire with a question such as, “How 
often do you use a napkin when you eat?” elicited responses ranging from 
neivous and perhaps less dian honest “very often” replies to die more honest 
but less helpful “I don’t know, I don’t count.” 

Field notes are ideally massive and detailed notes taken while die observa- 
tion is in process or at the first available free moment unobserved in die case of 
complete paiticipants or participant-observers. A detailed observation in the 
Mie village might be somediing like: 

5:08, Wednesday morning, week 23, the Tanaka husband enters the kitchen from 
the bedroom barefoot in his wrinkled pyjama bottoms. He is the first person to 
leave the bedroom this morning. He crosses the floor in three shuffling steps, turns 
on the rice cooker sitting on the table with a press on a single lever and yawns 
widely. Husband notices I am awake and smiles widely. Says “It’s early, isn’t it.” I 
agree with a nod. Husband scratches self in the right ribs. Husband nods four deep 
nods, bows in my direction, crosses the kitchen to the bedroom door and appar- 
ently returns to bed. 

Such detailed notes might be accompanied by a floor plan of the Tanaka kitchen. 
They might be full of abbreviations such as “TH” for Tanaka Husband, “4 n, 1 
db” for four nods and one deep bow. Also, good field notes contain parentlieti- 
cal-notes such as (wide-smile appears to be one of discomfort ratlier tlian friendly 
recognition), (all family members practice shuffling steps when anyone in the 
house is presumed to be asleep), (right ribs seem to be the preferred spot of 
self-scratching of TH whereas left ribs seem to be the preferred spot of nervous 
self-comforting of Tanaka son, daughter, and visiting cousin). 

The teacher of tlie Osaka college saidents had difficulty teaching her students 
to differentiate between taie observations and conjectures, a problem which often 
resulted in incomplete notes. Students had to be shown again and again to place 
parentlieses around their own opinions and to keep tliem separate and tieat them 
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differently from the obsei*ved data. “He wiped Iiis hands on his pants” is observa- 
tion, whereas “He is unclean” is parentlietical opinion. Also, parentlietical state- 
ments had to be expanded back into observations. The observer wrote, “She was 
happy when he touclied her hand,” to which die teacher asked, “How exactly did 
you know she was happy? Did she smile? Did she lean forward? Did her eyes 
narrow or widen? Did her breadiing or facial coloring change? Did she say, ‘I am 
happy?’” Edinograpliic obsei-vation, like all skills, improves widi practice. 

Tape recorders and video cameras are thought by some to have eased die 
necessity for detailed field notes. But there are fields of study which lack both 
available electricity or a supply of batteries for outdoor observers, and there are 
fields where movement, action, concealment, and privacy make these devices 
too unwieldy. Even when a video-camera has been in place, however, the 
obseiwer’s field notes are necessary. The human neck swivels to catch the small- 
est peripheral action, the human eye moves to change focus much more reac- 
tively with die human event than a camera on a tripod. The discomfort diat 
subjects feel when under the camera’s eye and how that effects the naturalness 
of their behaviors is also an important consideration. 

One teacher studying classroom action in Nagoya gets around the problem 
of subject awlcwardness by bringing two video cameras to class from die begin- 
ning of die semester. Students are asked to ham-up their self introductions, 
which are videotaped. Toward the end of die collegiate year, apparently, the 
students are so desensitized to the camera’s presence that the teacher can zoom- 
in on the most private of pairwork conversations with impunity. 

During fieldwork, the Mie researcher often perused his field notes to see if 
some patterns of behavior or communication where forming or if some interest- 
ing and unusual paths of inquiry have been overlooked. In the case of the 
shorter studies, a researchers can use their field notes to find points worthy of a 
further study or areas they have neglected to complete in the current study. 

4. Follow-Up 

Evans-Pritchard (1951), who did andiropological studies in foreign countries 
such as Sudan under die most aisdc and difficult of conditions in die 1950s, felt 
diat die most difficult aspect of ethnographic study was wridng-up die notes. 
Certainly injuries, illnesses, and other discomforts are part of the quality of being in 
the field; but it is in die wridng-up of field notes that real mental difficuldes arise. 
Not only is it a gargantuan task, but unlike the social task of being out and observ- 
ing your subjects, it is a lonely one usually spent at a desk widi your field note 
journals piled around your elbows. One cannot merely type it up and produce a 
treatise on what happened and dien what happened next. To make field notes 
readable die researcher must dig dirough die mountains of notes or tape record- 
ings, mine out the valuable insights, and present them in a natural setdng. The 
“mining tools” are die researcher’s own deep understanding of die observed tribe. 
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Ai*e There Special Considerations for Ethnography in Japan? 

If you are not a native Japanese, observing Japanese subjects has a possible 
problem with cultural misunderstanding and thus misinterpretation in the re- 
search. Even if you are fluent in the Japanese language, there may be some 
cases where intei’views or overheard conversations are not clear to you. So 
much of the fashion and customs of my own country, tlie USA, can be seen here 
in die large and modern cities of Japan that it is easy to forget that they are a 
facade over a nation diat is unique in values, attitudes, and heritage. I believe it 
may actually be easier for an edinographer working in a mountain village in 
Papua New Guinea to keep in mind the basic rule of “Watch and listen care- 
fully, for you are a stranger here and anydiing and everything is important!” In 
Japan, where diings seem to be easy to understand and one 'does not need to 
cany a big stick to fend off dogs, the observer may become less cautious than 
an edinographer must be. Too many important points may be missed. 

Here, too, one must make even greater efforts to ensure that a friendly native 
keeps an eye on how your observations are interpreted. Many Japanese people, 
however, who are friendly enough to help with your ethnography will not be 
culturally disposed to frankly criticize your mistakes or misunderstandings. 

There are, also, specific ediical considerations in this field. Many foreigners in 
Japan are here as English teachers and using your students as reporters and re- 
search-aids is an understandable choice. But, one must question whether taking 
class-time to teach ethnography tecliniques and to process interviews and obser- 
vation data is an appropriate use of English study time. The students in the Osaka 
example above had signed up, not for a regular English conversation course, but 
for a special course tided “Intercultural Communications and Research.” 

Japan does not, apparendy, have laws concerning privacy and protection of 
humans in experimental studies. Without the rigour of a review board, you 
must be more than conscientious in the way you set up, practice, and later 
publish your research. The edinographer works widi people, and must never 
forget diat people can be harmed. 



Conclusion 

There are just about as many edinographic styles of research as diere are ethnog- 
raphers, and one must clioose a mediod and refine it in the field to fit one’s own 
natiire and circumstances. All expatriates in Japan and their students may really 
find it fun to look around them and see all that is to be seen in this tribe. The 
follow-up publishing of one’s edinography in computer postings, professional 
journals (see Appendix), or in film documentaries must be done with ediical con- 
siderations but it can be fun as well. 
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Appendix: Some Journals that Publish Ethnographic Studies 



Anthropology/Sociology Communication Education/Language 



American Anthropologist 
American Ethnologist 
Anthropological Quarterly 
Cultural Anthropology 
Current A ntbropology 
Ethnology 
Family Process 
Human Studies 
Journal of Contemporary 
Ethnography 
Journal for the Theory of 
Social Behavior 
Journal of Applied Anthr. 
Man 

Qualitative Sociology 
Sociological Quarterly 



Communication Theory A nth ropology a nd Education 

Critical Studies in Mass Comm Quarterly 



Cultural Studies 
European Journal of Comm 
Journal of Comm. Inquiry 
Media, Culture & Society 
Research on Language & 
Social Interaction 
Semiotica 
Sigm 

Studies in Visual Comm. 

Symbolic Interaction 

Text & Performance Quarterly 



Discourse Processes 
International Journal of 
Qualitative Studies in Ed. 
International Journal of 
Sociology of Language 
Language in Society 
Public Culture 
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A language survey is any procedure which gathers information about partici- 
pants’ characteristics and views on tlie nature of language or language learning 
through the use of oral interviews or written questionnaires. In this paper, I will 
begin by examining tlie different types of surveys, including both interviews 
and questionnaires. I will tlien discuss how language surveys can be used for 
language curriculum development and research. I will continue by exploring 
tlie specific functions that surveys can perform. The next two sections will 
provide a set of guidelines for writing effective survey questions along with 
discussion of tlie issues involved in achieving a good return rate. I will then 
present a brief section on analyzing survey results and will end the paper with 
suggestions for further reading. 



Types of Surveys 

As shown in Table 1, surveys can take two basic forms: interviews (including 
individual, group, and telephone interviews) and questionnaires (including self- 
administered or group administered questionnaires). Each of these categories 
and subcategories merits a bit more discussion. 
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Table 1 

Types of Survey Procedures 



Type of Survey 


Procedures 


Inteiviews 


Individual 




Group 




Telephone 


Questionnaires 


Self-administered 




Group-administered 



Interviews 

Interviews are procedures forgathering information orally in specific, planned 
categories (if tlie interview is weU structured in advance) as well as infonnation 
tliat was not anticipated at tlie outset. Interviews can be conducted in three 
ways: individually, in groups, or by telephone. 

Individual interviews are interviews conducted witli participants on a one-to- 
one bases. Such interviews gather information privately, so a certain level of con- 
fidentiality can be maintained. As a result, individual interviews have tlie advan- 
tage (over group inteiviews) of collecting something close to the true views of the 
participants. However, such interviews also have the disadvantage of being rela- 
tively time-consuming. As a result, individual interviews may be most effective for 
finding out wliicli questions, views, or issues are wortliy of later follow-up study 
using more stmctured group interviews or questionnaires, both of which can be 
administered much more efficiently than individual interviews. 

Group interviews, which are interviews conducted in a group or meeting for- 
mat, can be used to circumvent the time-consuming nature of individual inter- 
views. However, group interviews also have the disadvantage of lacking confi- 
dentiality. For person^, political, or emotional reasons, the opinions expressed 
when participants are in a group interview may be quite different from the views 
expressed when they are interviewed individually and confidentially. 

Telephone interviews sometimes provide a useful compromise in that they 
are confidential and somewhat less time-consuming (saving the time neces- 
sary to travel to the participants). However, there are also several disadvan- 
tages that must be considered in using this type of interview: (a) in a tele- 
phone interview, some information may be lost because the interviewer can- 
not see the facial expressions, gestures, and surroundings of the participants; 
(b) a telephone survey also precludes the possibility of showing participants 
anything during the interview; and (c) at least in the United States, many 
people have become wary of telephone sales promotions and surveys and 
may not cooperate. 
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Questionnaires 

The various types of inteiviews can clearly be used to explore the questions, 
topics, views, opinions, etc, of participants in a particular language teaching 
institution. However, if tliese points are worth pursuing on a larger scale, ques- 
tionnaires may prove much more efficient, A c\uesXionnaire is any written in- 
stalment that presents participants with a series of questions or statements to 
which tliey should react either by selecting from existing possibilities or writing 
out tlieir answers. Questionnaires are particularly efficient for gathering infor- 
mation on a large scale. For example, an interview with a few of the participants 
in a language program might be helpful in designing a questionnaire, but the 
questionnaire itself would be much more practical to administer to all of the 
participants. Most questionnaires are tlie self-administered type. A self-adminis- 
tered questionnaire is usually mailed out and filled out by the participants in 
tlieir own homes or offices (i,e,, tliey are self-administered) and then returned 
by mail. Self- ad ministered questionnaires have three potential problems; (a) 
tliey often have a veiy low return rate, (b) they must be completely self-ex- 
planatoiy because further clarification is not possible, and (c) they are adminis- 
tered under conditions that are unknown to the survey designers. A group- 
administered questionnaire is any questionnaire which is administered to par- 
ticipants while they are in class or at a meeting. For example, if I wanted to 
suivey a particular group of students, I could go to their classes and ask their 
teachers to let me use tlie 15 to 20 minutes necessary for the students to fill out 
my questionnaire. By using such a strategy, I would solve all three of the prob- 
lems listed in tlie previous paragraph for the self-administered questionnaires: 
(a) tlie students will be a captive audience and will feel obliged to fill out the 
questionnaire (tlius giving me a high return rate), (b) I can be present to explain 
any ambiguities as they arise, and (c) I will know exactly what conditions ex- 
isted when tlie questionnaires were filled out. 



Uses for Language Surveys 

Surveys are most commonly used in language education for curriculum devel- 
opment and for research. Language programs provide a service to students who 
want to learn the language in question. If tlie program is interested in what 
tliese clients tliink, then some form of curriculum analysis and research will 
probably be desirable because tlie best way to find out what people are think- 
ing about any aspect of a language program is to ask them. 

Surveys for Curriculum Development 

The two elements of curriculum development wherein survey research is most 
useful are needs analysis and program evaluation (for more details on what all 
of these elements are, see Brown, 1989, 1995a). While surveys can be used to 
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investigate many elements of a curriculum (including objectives, tests, materi- 
als, and teaching), it is needs analysis and program evaluation that most often 
require sui*veys. 

Needs analysis. Needs analysis in language programs is often drought of 
asdie study of die language forms students will need to use in die target lan- 
guage when diey actually try to communicate. The focus is usually on the 
learners, and dieir needs are most often stated as linguistic needs. In fact, focus- 
ing on die learners in a needs analysis seems reasonable. After all, they are the 
clients; and die needs of die clients should be met. However, tliis view may be 
shortsighted because teachers, administrators, parents, employers, institutions, 
and even nations may also have needs that should not be overlooked in doing 
a needs analysis for a particular language program. Needs analysis should prob- 
ably focus on the language needs of students, but must recognize that students 
also have odier needs, called in Brown (1995a) “situational needs.” Situational 
needs include all of die learners needs as human beings. This means that the 
definition of needs analysis should be fairly broad. For instance, needs analysis 
might be defined as die systematic collection and analysis of all relevant infor- 
mation necessaiy to meet the language learning needs of die sai dents within 
the context of the particular institutions involved in the learning and teaching 
situation. Since heeds analysis relies on input from a variety of different groups 
about the students’ linguistic and situational needs, survey projects may prove 
particularly useful. In fact, in gadiering information from the various groups of 
people involved, survey interviews and questionnaires may become inevitable. 
After all, one of the most positive features of survey research is diat it can be 
used to efficiently gather a wide variety of different types of information from a 
wide assortment of sources. 

Program evaluation. Program evaluation might be defined as the systematic 
collection and analysis of all relevant information necessary to promote the 
improvement of the curriculum and assess its effectiveness widiin the context 
of the particular institutions involved. Since this definition is very similar to the 
one given above for needs analysis, the reader might correctly conclude that I 
view the program evaluation process as a kind of ongoing needs assessment. 
That is true. However, program evaluation is typically based on considerably 
more and clearer information dian needs assessment. While needs analysis is 
usually done in die early stages of curriculum development and must depend 
on interview procedures, questionnaires, linguistic analyses, conjecture, and a 
good deal of professional judgment, program evaluation can take advantage of 
all of the above information and tools to assess die effectiveness of a program 
but can also utilize any other information gathered while developing program 
objectives, developing and analyzing tests, creating materials, and/or teaching 
the curriculum. Program evaluation, then, will be defined here as the ongoing 
process of information gathering, analysis, and synthesis, the entire purpose of 
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which is to constantly improve each element of a curriculum on the basis of 
what is known about all of the other elements, separately as well as collec- 
tively. Thus, survey research is a caicial part of program evaluation just as it is 
for needs analysis and for precisely the same reasons. While survey projects 
used in curriculum development generally remain in-house, some find their 
way into print (for examples, see Kimzin & Proctor, 1986; or Richterich & 
Chancerel, 1978). 

Suweys for Language Research 

Surveys are also useh.il for other types of research not directly related to curricu- 
lum development. For example, surveys have been used to investigate the char- 
acteristics of graduate level teacher-training programs (Ochsnef, 1980), or the 
opinions and attitudes of members of a national professional organization (Brown, 
Knowles, Murray, Neu, & Violand- Sanchez, 1992). Of all the available research 
methods, sui’vey research may be tlie most practical and easy to do because it 
relies heavily on common sense and less on complex statistics. In fact, survey 
results are most often reported as simple percentages and averages. Since lan- 
guage survey research can be used to describe, explore, or explain physical 
characteristics, phenomena, behaviors, attitudes, etc., there is no shortage of 
useful questions that can be answered by doing survey research. Given tliat 
people’s ideas, atti aides, opinions, etc. change over time, tlie possibilities for 
doing language survey research seem inexliaustible because any research done 
today might usefully be repeated in five or ten years to see if the participants’ 
ideas, attitudes, opinions, etc. have changed over time. 



Steps in Survey Projects 

Tlie main problem in setting up a survey project is to make sure that it is 
planned carefully in a step-by-step manner so the project can be conducted 
systematically and effectively. Stacey and Moyer (1982) suggest the following 
ten steps for tlie constiuction of surveys: 

1. Specifying survey objectives and research questions. 

2. Reviewing the literature. 

3. Defining abstract concepts. 

4. Selecting question formats. 

5. Selecting tlie statistical analysis. 

6. Writing the sui-vey questions. 

7. Ordering the questions in the survey. 

8. Adjusting tlie physical appearance of the survey. 

9. Preparing tlie cover letter and instructions. 

10. Validating tlie suivey. 
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These ten steps are fine if the only concern is with creating a survey instrument. 
However, I feel that a language survey instRiment should be viewed as part of 
a larger survey project, one which involves die following steps: 

A. Planning the survey project. 

B. Designing the sui-vey instrument. 

C. Gathering and compiling the survey information. 

D. Analyzing die survey statistically. 

E. Analyzing the sui-vey logically. 

F. Reporting the results of a survey. 

If each of these six steps is carefully carried out (along with the ten steps for 
constructing a sui-vey proposed by Stacey and Moyer (1982) under point B of 
my list), any sui-vey project will stand a much better chance of success. 



Functions of Surveys 

Anyone designing a suivey instniment must realize diat surveys can function in 
different ways. For instance, surveys can function as: biodata surveys, opinion 
surveys, self-ratings, judgmental ratings, rankings, or Q-sorts, Obviously, some 
or all of these functions can be combined in one interview or questionnaire. 

Biodata suweys are used to elicit biographical background information about 
die participants, including diings like the participant’s age, place of birth, sex, 
marital status, occupation, number of years of language study, years of educa- 
tion, major field of study, and salary. 

Opinion surveys are designed to uncover the opinions and attitudes of the 
participants about specific issues. For instance, questions might be developed 
to find out what a group of students think (i.e., dieir opinions) about the goals 
and objectives of a particular language program as well as about its tests, mate- 
rials, teacliing, administration, etc. 

Self-ratings require the participants to rate their own abilities, interest levels, 
motivations, etc. Such ratings reveal information about die self-image of the 
individuals involved. 

Judgments can be used to obtain the views of participants about various 
aspects of language learning. Judgments can be obtained eidier by asking par- 
ticipants to briefly write their views down or by using a more structured type of 
question. For example, students might be asked to judge the effectiveness of a 
language program’s objectives and materials in terms of how useful they are, 
how necessary diey are to future language use, or how easy they are to learn. 
The students would simply be providing their judgments. 

Rankings require the participants to order some set of words, or phrases, 
from most important to least important, or most useful to least useful, etc. For 
instance, students might be asked to rank a list of the 15 types of classroom 
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activities used in a particular course from most enjoyable to least enjoyable by 
numbering them 1 to 15. 

Q-sorts combine attributes of one of tine earlier survey types with the rank- 
ing procedures discussed immediately above. Tliis combination is accomplished 
by: (a) having the participants give their views, attitudes, opinions, self-ratings, 
and/or judgments; (b) then asking the participants to rank tine issues in terms 
of how important they think each is. While survey instruments can be designed 
to perform any one of the six functions discussed above, they are seldom so 
unidinnensional; surveys are most often developed to accomplish a combina- 
tion of functions- 



Guidelines for Writing Good Survey Questions 

Once decisions have been made about which hmctions to use in a survey (or 
which combinations of functions), the actual questions must be written. Writ- 
ing good questions can be surprisingly difficult. Guidelines for writing good 
survey questions are given in Table 2. Overall, the goal in writing survey ques- 
tions should be to make them so clear and precise that the participants will 
know exactly what is being asked of them. 



Table 2 

Things to avoid in writing good survey questions 



Avoid: 

1- Overly long questions 

2. Unclear or ambiguous questions 

3. Negative questions 

4. Incomplete questions 

5. Overlapping choices in questions 

6. Questions across two pages 

7. Double barreled questions 

8. Loaded word questions 
9- Prestige questions 

10- Embarrassing questions 

12- Biased questions 

13- Questions at the wrong level of language 

14. Questions that respondents are incompetent to answer 

15. Assuming that everyone has an answer to all questions 

16- Making respondents answer questions that don’t apply. 

17- Using irrelevant questions 

18- Writing superfluous information into questions. 
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Notice tliat Table 2 is cast in negative terms because it is a list of tilings tliat 
should be avoided in writing survey good survey questions. To begin with; (1) 
overly long questions, (2) unclear or ambiguous questions, (3) negative ques- 
tions, and (4) incomplete questions should all be avoided. Overlapping choices 
(5) should also be avoided. The issue of overlapping choices arises when op- 
tions are not mutually exclusive. In the following example, tlie options are 
overlapping in die sense that Santa Ana High School is located in tlie city of 
Santa Ana and in Orange county so the options are not mutually exclusive: 

The best location for the new foreign languages building would be: 

In Orange County 

In die city of Santa Ana 

In Santa Ana High School 

Other: 

Questions should also be typed so they are located in one compact place on 
a single page (6) so diat participants will not miss part of die question. Double- 
barreled questions (7) arise when a question is really asking more than one 
question. For instance, the following example contains three questions that 
could be asked separately (one about education spending, one about military 
spending, and one about language education spending): 

The United States should spend more on education (especially 
language education) and less on die military. 

Agree Disagree Don’t know 

Loaded words (8) are words that suggest an automatic positive or negative 
response because diey are emotionally charged. For instance, in the following 
example, the word obviously v>jou\d be a loaded word: 

Spanish is obviously the language to study if you live in the 
Soudiwestern United States. 

Agree Disagree Don’t know 

Leading questions (9), namely, questions that overdy encourage participants to 
answer in a certain way, should also be avoided. It is also important to avoid 
questions diat may contain issues of prestige (10). For example, when asked, some 
people will claim diat diey read more than they do or that they speak a language 
better dian diey do because diat is the prestigious way to answer. Embarrassing 
questions (11), or questions that a particular group of participants miglit fmd em- 
barrassing (e.g., questions containing swear words) should be avoided, as should 
questions diat show prejudice or bias (12) against a particular group of people. 

In addition, questions at the wrong level of language (13) and questions that 
participants are incompetent to answer (14) should be avoided. And, as matters 
of sound survey development policy, it is probably best not to (15) assume that 
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eveiyone has an answer to all questions, (l6) make participants answer ques- 
tions that don’t apply to tliem, (17) use irrelevant questions, or (18) write super- 
fluous information in questions. 



Guidelines for Producing Effective Questionnaires 

Writing good questions, as covered in the previous section, is just one of the 
issues involved in producing sound surveys. Many others are covered in Brown 
(unpublished ms.) 



Table 3 

Guidelines for Producing Effective Questionnaires 



1. Write good questions (see Table 2) 

2. Order the questions rationally 

3. Format the questionnaire for clarity 

4. Write clear directions 

5. Edit carefully 



For questionnaire surveys, Table 3 suggests tliat tlie ordering of questions is also 
important. In order to make a questionnaire clear to tlie participants, it may help to 
rationally group the questions by tlieir type, function, format, topic, or some com- 
bination of tliese factors. (For more ideas on structuring surveys, see Gallup, 1947). 

Formatting the questionnaire for clarity is anotlier important issue, wliich 
may involve effective use of spacing, type faces, graphics, higlilighting, etc. 
Writing clear directions is another crucial issue to tliink about because unclear 
directions can confuse and discourage participants. Last but not least, it is nec- 
essary to use careful editing, perhaps including getting others to edit tlie ques- 
tionnaire, piloting it with a few students, and doing very careful final editing. 



Guidelines for Increasing Mail Survey Return Rates 

There are also a number of ideas that might help in getting a relatively high 
return rate, especially for those who are using a mail survey, where a question- 
naire is mailed out and participants are asked to send it back after filling it out. 
First, a cover letter explaining die background and importance of the question- 
naire may help. Participants are also more likely to return a short questionnaire 
that is easy to fill out than to return a long and difficult one. If possible, some 
sort of incentive should be offered to participants for answering the question- 
naire (e.g., a monetaiy reward, or a copy of die study that will result from the 
questionnaire). Providing return postage and a self-addressed envelope will 
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also encourage participants to return die questionnaire. It is also a good idea to 
put die return address somewhere else on the questionnaire so it can be sent 
back even if the participant loses the envelope. 

For participants who do not return die questionnaire, a follow-up letter or 
phone reminder may encourage diem to complete the task. The dining of the 
mailing may also be significant. For instance, if a questionnaire mailing goes out 
late in die school year or arrives during die summer break, it is unreasonable to 
expect academics to respond quickly in great numbers. 



Tabulating and Analyzing Survey Results 

Once die suivey results have been collected, some system must be worked out 
for tabulating and analyzing all of the data. The system will differ for closed- 
response questions and open-response questions (bodi of which may be found 
on die same questionnaire). 

Closed Responses 

Closed-response questions are diose wliich offer die participants options from 
wliich to select dieir answers. Tliis category includes Likert scales (the 1 to 5 type), 
yes/no questions, or any odier select-a-response type question. Such questions 
can be analyzed with stmiglitfoiward averages or percentages of people selecting 
each option, or bodi. For die statistically initiated, standard deviations and clii- 
squared statistics may prove useful, but the averages and percentages are usually 
sufficient for finding interesting patterns in closed-response survey results. 

In addition, it is often a good idea to check the reliability of a survey instru- 
ment — especially if it is being used to make important decisions or to do pub- 
lishable research. Given diat diere are often weighted scales on questionnaires, 
die Cronbach alpha reliability estimate is probably the best to use. Cronbach 
alpha is available for use on MS-DOS system computers in die SPSS/F6’+’''^ 
(1990) or (Wilkinson, 1988) software, or it can be calculated by hand 

using the forniula found in Brown (1995b). 

If there are subscales involved in the survey, it may be a good idea to exam- 
ine the reliability of each subscale as well as the combination of all closed- 
response subscales taken together. 

Open Respomes 

Open-response questions are questions that require die participant to produce 
an answer, in either an oral or written fonn. If die responses are oral, they 
should be taped in the process of gathering die responses. It is usually neces- 
sary or at least advisable to transcribe the tapes to a written form. 

Even if die open responses are written by die participants, I have usually found 
it necessaiy to enter diem into computer files for easier analysis. I then do com- 
puter analysis of diese files using a software package like ^ghtWriter^^ (Que 
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Software, 1990) to get frequenq^ counts of the words in the files. Sometimes, as 
part of die analysis, I use die search funcdon in my word processor to searcli for 
die most commonly used words and read how die respondents used diem. 

With or widiout die technological boost diat computers provide, the open 
responses should all be read, analyzed, sorted, and summarized in some way. 
This process may involve simply putdng diem on cards and getdng down on 
die floor to sort them out, or may involve high technology. Eidier way, the 
human brain ultimately ends up doing the analysis and interpredng the results. 
For examples of interpretadons of such open-response quesdons, see Brown 
(1992a, 1992c, 1992d, & 1992e). 



Suggestions for Further Reading 

A number of different topics have been covered in this chapter. Naturally, each 
of diese topics is covered in more detail elsewhere. For instance, more informa- 
don on the different types, uses, and funcdons of surveys, as well as ideas for 
wridng good survey quesdons and getdng good return rates can be found in 
Babbie (1973); Bailey (1982); Brown (unpublished); Oppenheim (1966); or Stacey 
and Moyer (1982). Examples of how surveys have been used for language 
curriculum development (especially for needs analysis and program evaluadon) 
are available in Brown (1995a); Kimzin and Proctor (1986); Loschky, Stanley, 
Cunlia, and Singh (1987); Richterich and Chancerel (1978); and Weaver, Pickett, 
Kill, and Cook, J. (1987). 

Examples of how suiveys have been used in language research can be found 
in Bailey and Brown (1995); Brown (1992a, 1992c, 1992d, 1992e, and 1995a); 
Brown, Knowles, Murray, Neu, and Violand-Hainer (1991, 1992); Johns (1981); 
Miller (1995); and Ochsner (1980). 

More details on using computers for program level decision making and 
research can be found in Brown (1992b). 

Additional information on the stadsdcs necessary to analyze closed-response 
survey results, see Brown (1988); Buder (1985); Hatch and Lazaraton (1991); 
Seliger and Shohamy (1989); and Woods, Fletcher, and Hughes (1986). 

More details on increasing die reliability and validity of the open-response 
analytic processes are available from Agar (1986); Davis (1992); or Kirk and 
Miller (1986). 

Survey interviews and quesdonnaires provide seemingly simple and straight- 
foiward tools diat can be used effectively for curriculum development or other 
types of research. However, language professionals who actually use diese tools 
often find diat diey are not as simple as they seem. Hopefully, this paper has 
provided much of the information necessary to effecdvely plan and conduct 
surveys, or at least pointed the reader to sources that can help them use surveys 
responsibly and professionally. 
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Bare Minimum Knowledge 
for Understanding 
Statistical Research Studies 
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This article is for those language teachers who find reading statistical researcli 
troublesome, but necessary. First it will examine the statistical reasoning under- 
lying quantitative, empirical studies. Normal distribution, standard deviation, 
three indices of central tendency (mean, median, and mode), and skewness will 
be discussed. Then, I will briefly look at tlie three different kinds of compari- 
sons most frequently reported in quantitative studies, namely, comparisons of 
means, frequencies, and correlation coefficients. Statistical tools typically used 
for each comparison will be touched upon. In the course of the discussion, 
useful checkpoints will be provided that readers of statistical research studies — 
not to mention researchers themselves — should be equipped with. 



Normal Distribution 

One of the most interesting phenomena that occurs and recurs in nature, is a 
pattern called normal distribution (also called bell or normal curve). Suppose, 
for instance, if you decided to plot on a chart the lengths of a bagful of bean 
sprouts, tlie result might look sometliing like the one in Figure 1.1. The shape is 
not yet “normal," but the larger the number of sprouts you measure, the more 
“normal” the form of the scatterplot becomes, and it will evenrtially look some- 
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tiling like tlie frequency polygon shown in Figure 1.2. As a language teacher, 
you might want to plot your students’ test scores. In tliis case, most probably, 
you would also find something close to a normal distribution among your stu- 
dents when the number of tliem is sufficiently high (usually, at least 30). 
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Central Tendency 

Let us now briefly review tliree indices of central tendency: tlie mean, the 
mode, and the median. Tlie mean is tlie aritlimetic average, and is tlie most 
commonly reported indicator of central tendency. The mode is, the most fre- 
quent score in a set of data. The median is tlie score wliich divides the entire 
data in half. If tlie mean and the median coincide, and if such a distribution has 
a single mode, the frequency distiibution will show an ideal symmetrical curve. 

What is intriguing about such normal distribution is that we would expect 
about 68% of all tlie scores to fall within one standard deviation of tlie mean, 
and 95% to fall within two standard deviations (see Figure 1.3 above). Standard 
deviation shows dispersion, or in otlier words, how much the scores vary away 
from, or spread out around, the mean (Brown, 1991, p. 574). 



Figure 2.1 A Symmetrical Distribution 




Figure 2. 2 A Magatively Skewed Distribution Figure 2. 3 A Positively Skewed Distribution 
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Table 1 Descriptive Statistics for Comprehension 



Group 


m 


SD 


min 


max 


D 


G1 


3.80 


2.20 


.0 


10.0 


75 


G2 


2.76 


1.67 


.0 


9.0 


75 


G3 


4.79 


2.47 


.0 


13.0 


75 


G4 


4.04 


1.88 


.0 


10.0 


75 



Tliis normal disaibution, in its perfect form, however, rarely occurs in the field 
of language teacliing unless die number of subjects is extremely large. Tlie values 
of die measures of central tendency usually differ. When die value of mean is 
influenced by die size of exti'eme scores, it is pulled toward eidier end of die 
distribution in wliich die extreme scores lie, as shown in Figures 2.2 and 2.3, and 
die distiibution will show an asymmetrical curve (described as being skewed). 

When we read statistical studies, we should look to see how far the distribu- 
tion is skewed from the normal curve, because statistical analyses are based on 
normal distribution. If you find the distribution is extremely skewed (not close 
to normal), you should look for die kind of interpretation diat is made by die 
researcher of the study. The skew of a distribution can be identified by compar- 
ing the mean and the median without necessarily constructing a liistogram or 
frequency polygon (see Figures 1.1. and 1.2 for examples). When the distribu- 
tion is skewed toward die lower end, or negatively skewed, the mean is always 
smaller than the median, and the median is usually smaller than the mode (see 
Figure 2.2). When a distribution is skewed toward the higher end, or positively 
skewed, the mean is always greater dian die median and the median is usually 
greater than the mode (see Figure 2.3). Unfortunately, most statistical studies do 
not provide information on die median. Thus, we usually have to turn to other 
indices usually provided in the table called “descriptive statistics.” Table 1 is an 
abbreviated version of one such example. It displays such vital information as 
the number of students involved in die study (n), the mean (m), the standard 
deviation (SD), and minimum (min) and maximum (max) scores. 

As is shown, four groups (G1 to G4) each consisting of 75 subjects took a 
comprehension test of some kind, and their scores ranged from zero to 13. The 
means (m) for all groups are pulled slightly towards the lower ends. We know 
this because all mean scores are lower than the halfway point scores between 
the lowest possible scores of zero and the highest possible scores of ten, nine, 
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thirteen, and ten for each group, respectively. However, the scores of each 
group are spread out to a reasonable degree (we know this because tliere is 
room for two standard deviations above and below die mean within the range 
of the lowest possible and highest scores), therefore, we can conclude diat the 
skewness is not a serious problem here. The table also shows that in terms of 
means. Group 3 comes first, and Groups 4, 1, and 2 follow. The standard 
deviation (SD) for Groups 1 and 3 are liigher, however, indicating more spread 
or greater dispersion. 

What we would like to know, dien, is whether these mean differences are 
significant. Can we confidendy say diat they are really (statistically) different^ 
In other words, did the differences we observed happen accidentally, or did 
they manifest themselves because of some other systematic factors? In order to 
answer tliis question, researchers conduct statistical analyses. (For a clear and 
concise account on the logic of statistical inference behind these analyses, see 
Nunan, 1992, pp. 28-37). 



Three Different Kinds of Comparisons 

The tlii*ee most common types of statistical tests typically reported are mean 
comparisons, frequency comparisons, and comparisons of correlation coeffi- 
cients to zero. Let us look briefly at each of them. 

Mean Comparisons 

A mean comparison is comparing two or more groups by comparing their 
average scores on some test. There are four kinds of statistical tests typically 
used widiin this categoiy. They are t-test, one-way AN OVA, n-way ANOVA 
(factorial design), and MANOVA. 

Tlie t-test is used to compare the means of two groups, whereas one-way 
ANOVA is used to examine die differences in more than two groups. N-way 
ANOVA enables us to analyze die effect of different treaUnents in more complex 
conditions, such as different proficiency levels, or different types of learners 
(e.g., learners widi different learning styles). Therefore, die comparisons are 
drawn in more than two directions (actually, N directions) in n-way ANOVA. In 
odier words, there are more than two (N) independent variables. However, 
ANOVA is just an exploratory tool, wliich only tells the researclier that there are 
some significant differences somewhere in the data but does not specify where 
diey are. Subsequent analyses are needed for the researcher to identify where 
the significant differences lie (see die one-way ANOVA section below). Still, 
however, no information can be obtained about the magnitude of the effect. 

Diagramatic representations of each of the tests are on the following page. 

Now, let us use die data in Table 1 and actually mn these statistics to see if 
we can make any sense out of the computer printouts. 
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t-test 
G1 G2 



Qne-«ar ANOTA 
«=> 

G1 G2 G3 



Two^y mu 
«=> 

G1 G2 G3 . 



1 

Gender 

F 



Tliree-vay ANOTA 
«=> 

G1 G2 G3 




Diagrama^ic representations of t:ypes of designs 
appropriate for t-^test« oneway ANOVA^ and n-^ay ANOVA 



T-test: For mean comparison between two groups. For the sake of simplicity, 
let us assume tliere are only two randomly chosen groups (groups 2 and 3 in 
Table 1) learning a set of new vocabulary, and we are comparing the perfor- 
mances of these two groups. Group 2 learned a set of vocabulary using only a 
listening task, whereas Group 3 learned, the same set tlirough the same listening 
task, but with the help of written input. After the task, we investigate the effect 
of the written input by giving both groups the same vocabulary test. 

Below is a computer printout (SPSS/PC+) of the t-test results. It shows the t- 
value to be -5.89, wliich is significant at the .05 level. This means that Group 3 
subjects who learned a set of vocabulary through a listening task with the help 
of written input learned better than those in Group 2. This will be reported as: 
t= -5.89; p^ .05. This significance level (called alpha decision level) should be 
set at the beginning of the study. Usually, it is set at the conservative a<.01, or 
at the more liberal a<.05. 

One-way ANOVA: For comparing differences in more than two groups. Sup- 
pose tliere were four groups (groups 1, 2, 3, and 4 in Table 1). Each group was 
given a different kind of treatment, for instance, tliey were taught by using tlie 
G-T method, TPR, Silent Way, and Suggest opedia. Now, we want to know if 
these different treatments had any effect. Since the comparisons are made in 
only one direction, that of tlie independent variable, which is “method,” this 
ANOVA is called a “One-way” ANOVA. 

Below is a computer printout of die one-way ANOVA. F probability (.0000) 
indicates diat diere were significant differences between die four groups, but it 
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Independent samples of GRP 



Group 1: GRP EQ 2.0 



Group 2: GRP EQ 3.0 



t-test for: COMP 

Number Standard Standard 

of Cases Mean Deviation Error 





Group 1 


76 


2.7600 


1.667 


.193 






Group 2 


75 


4.7867 


2.468 


.286 








Pooled Variance Estimate 


Separate Variance Estimate 


F 


2-Tail 


t 


Degrees of 


2-TaU 


t 


Degrees of 


2-Tail 


Value 


Prob. 


Value 


Freedom 


Prob. 


Value 


Freedom 


Prob. 


2.19 


.001 


-6.89 


148 


.000 


-6.89 


129.91 


.000 



[SPSS/PC+ printout 1] 

does not specify where tliose differences lie. Also, although descriptive statistics 
(mean and standard deviation) provides an indication of where tlie differences are 
likely to exist (e g., of all tlie mean differences, tliat between G1 and G4 is tlie 
smallest), tliese insights must be cliecked statistically. Tlie post hoc Scheffe con- 
trast test does tliis job. As shown at tlie bottom of the printout, where an asterisk (*) 
denotes pairs of groups significantly different at tlie .05 level, differences exist 
between Groups 1 and 2, 2 and 3, 2 and 4, and 1 and 3, but not 1 and 4. 

Readers are advised not to use multiple t-tests here. Although there have 
been several studies using multiple t-tests published even in prestigious jour- 
nals in our field, using t-tests repeatedly (for multiple comparisons) is not rec- 
ommended because the more comparisons you make, the more chances of 
creating spurious results. 

N-way ANOVA: For examining the effect of several variables studied simulta- 
neously, as well as the interactions among the variables. As is shown 
diagramatically above, tlie comparisons are made in more tlian two directions 
(i.e., in N ways) in N-way ANOVA. (For a more detailed account, see Hatcli & 
Lazaraton, 1991, pp. 301-331.) 

MANOVA: For research designs including more than one dependent variable. 
Language studies often include two or more dependent variables which are 
related to each other. For example, if tliree different kinds of tests were given to 
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ONEWAY 



Variable COMP 
By Variable GRP 







Analysis 


of Variance 










Sum of 


Mean 


F 


F 


Source 


D.F. 


Squares 


Squares 


Ratio 


Prob. 


Between Groups 


3 


157.8000 


62.6000 


12.1909 


.0000 


Within Groups 


296 


1277.1467 


4.3147 






Total 


299 


1434,9467 









ONEWAY 

Variable COMP 
By Variable GRP 

Midtiple Range Test 

Scheffe Procedure 
Ranges £br the .050 level < 

3.98 8.98 3.98 

The ranges above are table ranges. 

The value actually compared with Mean(J) > Mean (I) is .. 

1.4688 ^ Range * Sqrt (1/N(I) + 1/N(J)) 

(*) Denotes pairs of groups significantly different at the .050 level 



ONEWAY 



Variable 


COMP 










(Continue cO 














G 


G 


G G 








r 


r 


r r 








P 


P 


P P 




Mean 


Group 


2 


1 


4 3 




2.7600 


Grp 2 










3.8000 


Grp 1 


* 








4.0400 


Grp 4 


* 








47867 


Grp 3 


* 


* 




[SPSS/PC+ printout 2] 
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LIFE Is life exciting or dull? by SEX Respondenta's sex 



Count 
Exp Val 
Residual 

LIFE 


SEX 

Male 

1 


Page 

Female 

2 


1 


300 


384 


Excited 


279.0 


405.0 




21.0 


-21.0 


2 


296 


481 


Mot excited 


317.0 


460.0 




-21.0 


21.0 


Column 


696 


865 


Total 


40.8% 


69.2% 


Cbi-Square 




Value 



Row 

Total 

684 

46.8% 



111 

63,2% 



1461 

100 , 0 % 



DF 



Significance 



Pearson 5. 00467 1 

Continuity Correction 4.76884 1 

Lilcelibood Ratio 5.00327 1 

blantel-Maenszel test fer 5.00124 1 

linear association 



.02628 

.02898 

,02630 

,02633 



Minimum Expected Frequency = 279.031 

dumber of Missing Observations: 12 



[Cbl-square printout! 



tlie same subjects in two different groups, the statistical test called for is not 
ANOVA, but MANOVA, because the scores from the three different kinds of 
tests administered to the same subjects should somehow be related. These 
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Correlations: FIN 



MLAT5 PLAB5 PLAB6 MUT4 COMP 



FIN 














MUT6 


.1736* 












PLAB6 


.1233 


.1346* 










PLAB6 


-.0231 


.0795 


.2317** 








MLAT4 


.0326 


.2718** 


.1071 


.2111** 






COMP 


.0033 


.3876** 


.1032 


.0083 


.2126** 




PTl 


.1424* 


.2582** 


.1615* 


-.0656 


.1241 


.4303** 


PT2 


.0656 


.2191** 


.0791 


.0250 


.1188 


.4474** 


PT3 


.1100 


.1199 


.1047 


-.0738 


.1787** 


.3083** 


N of cases: 


300 1-tailed Signif 


*-.01 


**..001 







Correlations: PTl PT2 PT3 



FIN 

MUT5 

PLAB5 

PLAB6 

MUT4 

COMP 

PTl 

PT2 .6597** 

PT3 .6137** .6634** 



N of cases: 300 l*tailedSigni£ *-.01 **-.001 



[SPSS/PC+ printout 3] 

related variables should be analyzed together, not separately. (For further de- 
tails, refer to Hatch & Lazaraton, 1991, pp. 386-387). 

Frequency Comparisons 

Chi-square: For comparing two nominal (frequency) data. Frequency means 
counting the number of times something happens. When examining relations 
between frequencies, die Chi-square analysis is the usual procedure employed. 
For example, suppose you conducted a survey asking people whether or not 
they think life is exciting. One thousand four hundred and sixty one persons 
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responded, and 50.3% of all the men (300 out of 596) and 44.4% of all the 
women (384 out of 865) described life as exciting. Is tliis evidence sufficient 
enough to believe that men differ from women in finding life exciting? To an- 
swer a question like tliis, the Chi-square test is calculated. 

As shown in tlie computer printout, this procedure compares the observed 
frequencies and expected frequencies to see if tlie former are greater than 
chance alone. The observed significance level of a Chi-square of 5.005 is 0.025. 
This indicates that a discrepancy this large between the observed and expected 
frequencies would occur only 2.5% of the time if, in tlie population, men and 
women are equally excited witli tlieir lives. Since tlie observed significance 
level is quite small, we can conclude that men and women are not equally 
likely to find life exciting. 

One impoitant fact we should bear in mind here is tliat tliis family of analy- 
ses should be used to research where only nominal variables are included and 
frequencies are compared, and that, more important, tlie Chi-square procedure 
does not allow us to make cause-effect claims. 

Comparisons of Relationship 

Pearson correlation: For examining existing relationships between variables 
without any manipulation of the variables (e.g., no treatment). For example, if 
we wonder whether a good language aptitude is related to success in vocabu- 
laiy learning and acquisition, we might want to know tlie relationship of the 
subjects’ scores on aptitude tests as well as tlieir scores on a vocabulaiy com- 
prehension and retention test. Below is the computer printout of one such 
study (Nagata & Ellis, 1996, Forthconiing), FIN, MLAT4, PLAB5, PLAB6, and 
MLAT4 are subtests of aptitude batteries, COMP, PTl, PT2, and PT3 are vocabu- 
laiy comprehension and retention tests. The asterisk (f) shows where significant 
relationships were detected. 

Here again, as witli the Chi-square analysis, we should be cautioned tliat tlie 
correlation tells us only that tliere is some degree of relationship between tlie 
two variables. We cannot, therefore, make any cause-effect claims. 



Statistical Assumptions to Remember 

Selecting an appropriate statistical test is a caicial point in any statistical study. 
Major questions we should ask ourselves are: (a) How many variables are tliere 
and what are tlieir functions in tlie study (independent vs. dependent variables)?; 
(b) Wliat types of measurement tools (scales) were used?, and; (c) Where do tlie 
data come from, i.e., from two different groups (a between-groups design), or two 
or more measures taken from the same group (a repeated-measures design)? 

Even after an appropriate test has been selected, we should not forget tlie fact 
tliat all statistical tests have certain assumptions underlying tlieir fomiulation, Wlien 
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even one of these assumptions is violated, the probability gets distorted, and it 
becomes difficult to know how much confidence to place in tlie findings. Check to 
see if tliese assumptions are met. If you are tlie reader of a statistical study, see if 
die researcher checked whether the basic assumptions of the specific test(s) em- 
ployed were met. (For a compact list of assumptions and solutions when these 
assumptions cannot be met, see Hatcli & Lazaraton, 1991, pp. 546-554.) 



By Way of Conclusion 

I have taken you quickly through the winding mountain path of basic statistical 

manipulations often used in our field. Just to recapitulate some of the check- 
points which have so often been neglected in the statistical studies published in 

our field: 

1. Check to see if the appropriate statistical tests are used. 

2. Check to see if all the statistical assumptions are met. 

3. See if the number of subjects is sufficient. (Rule of thumb says more dian 
30.) 

4. Check to see if multiple comparisons are being made erroneously. Espe- 
cially watch out for the use of the ordinary t-test for making multiple com- 
parisons). 

5. See if the nominal, frequency data are properly dealt widi, in other words, 
by the Chi-square analyses for independent data. 

6. Check to see if the researcher’s interpretation of the findings stays witliin the 
statistical logic. For example, die Chi-square procedure is a non-parametric 
test. It does not allow us to make cause-effect claims. Neither does the 
Pearson correlation. 

7. Check to see if the dependent variables in the study are related. If so, use 
MANOVA, instead of ANOVA. 

8. See if the data are from two different groups or are two or more measures 
taken from die same group. In other words, be careful about die repeated- 
measures designs. Data, then, should be examined in terms of widiin-subjects 
differences radier dian between-subjects differences. If, for instance, a pretest 
and a posttest are administered, die study is repeated-measures in design. 



References 

I would like to use diis section not only as a conventional reference section, but 
an introductory resource section. Therefore, the books and articles are not in 
conventional alphabetical order. The following are books and articles which I 
have found accessible, understandable, and extremely readable. If you are new 
to statistics you might like to start reading in die order they are presented here. 
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In-Depth Interviewing 
as Qualitative Investigation 



Marilyn Books 

Waseda University 



Intei-viewing is rather like marriage: everybody knows what it is; an awful lot of 
people do it, and yet behind each closed door there is a world of secrets (Oakley, 
1986, p. 231). 

Social behavior is complex and multidetermined, and there is no perfect method 
for studying it. Of the variety of methodological approaches, this paper presents 
in-deptli interviewing as a strategy in qualitative research for discovering tlie 
secrets of the complex human psyche. Krathwohl (1993, xvii) affirmed that 
most research texts: 

hew to a positivistic point of view, stressing experimentation and ignoring or mini- 
mizing qualitative research .... It is clear that we have entered an era where more 
than the positivistic paradigm is considered defensible. 

What determines the use of a qualitative metliod such as in-deptli interviewing?: 
the puipose of tlie saidy does. If tlie question is, “How does the degree of adop- 
tion of the American custom of explicitness correlate witli tlie length of overseas 
experience?” tlien a survey may be the best choice. If tlie scliolar is wondering, 
“How do Japanese professors speak with tlieir American cohorts in staff meet- 
ings?” tlien participant observation would be a likely teclinique. If tlie objective is 
to determine whetlier Japanese language lessons hasten the acculturation of for- 
eigners, tlien an experimental metliod may suit best; however, if the purpose is to 
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comprehend tlie meaning of people’s existence (“Wliat is it like talking to a Japa- 
nese person who is speaking indirectly? What is your experience witli tlial/them?”) 
tlien open-ended oral questioning is a useful avenue of study. 

Tliis paper begins by looking at tlie nature of in-deptli interviewing, situating it 
on tlie continuum of research methodology, and differentiating it from quantitative 
research. It tlien discusses tlie strengths and weaknesses of open-ended surveying. 
Sampling to determine inteiviewees will be discussed in tlie next section. Finally, 
some advice on tlie actual administration of tlie interview is offered. 



The Sample Study 

The sample study discussed in tliis paper was originally tlie basis of a doctoral 
dissertation in communication (Books, 1996). The topical question was: “What 
are the positive and negative experiences of North American native English 
speakers and native Japanese speakers who teach English at Japanese universi- 
ties in relation to tliese key communication styles; Americans’ assertiveness, 
explicitness, and independence; and Japanese indirectness, nemawashi (ground- 
work), and tatemae (stated position)?” 



Research Methods 

Research metliods fall along a continuum between quantitative and qualitative 
approaches. Quantitative methodologies generally use numeric data to portray 
phenomenon, seek to validate hypotheses, preplan with variables and concrete 
notions about how to explain occurrences, maintain controls (sometimes exten- 
sive), employ staictures (often highly constrictive), and are often controlled in a 
laboratoiy setting. 

Qualitative methods usually employ verbal data, begin with few or no pre- 
conceived notions, seek to comprehend and explain or describe phenomena, 
maintain little or no control and structure, explore with the aim of discovering 
what exists that is of importance in the situation, engage in a more natural 
function, and are carried out in natural environments. 

Some metliods span the continuum and can be, depending on the situation, 
either quantitative or qualitative. Methods which are generally quantitative and 
stress validation are: experimentation (complete witli dependent and indepen- 
dent variables), and model building and testing (using relationships as predic- 
tors). Further away from tlie quantitative pole are: meta-analysis (studying similar 
studies) and longitudinal studies (long-term studies of groups). Those which 
typically lean more to description and rely at least somewhat on explanation 
are: single-subject studies (long-term studies of individuals), evaluation meth- 
ods (determining efficacy), historical methods (examining personal and institu- 
tional documents), surveys (usually closed questionnaires), and classification 
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schemes (placing material into useful categories). Theory building (exceeding 
actual data) is mostly description and explanation. 

At tlie otlier end of tlie continuum is qualitative researdi. It must be noted that 
action research is not really a metliod, but a useful approach for using any of the 
available research metliods, and such studies are carried out in tandem with tlie 
benefactor, for example, tlie educator who can use the results. A more thorough 
discussion of the variety of research methods can be found in Krathwohl (1993). 

Subsumed under qualitative research are these fundamental strategies: observa- 
tion, participant observation, document reviews, and interviews. Observation in- 
volves noting and recording behaviors, events, and objects in the setting studied. 
Paiticipant obseivation immerses tlie researcher in tlie setting and experiences of 
those obseived. Included in document reviewing is the perusal of archival data. 

Patton (1990) noted tliat inteiviews vary in the determination and standardiza- 
tion of questions before tlie interview. He divided open-ended interviews into 
tliree categories: tlie informal conversational interview, the standardized open- 
ended interview, and die general interview guide approach. Tlie conversational 
style entails a spontaneous flow of questions. The standardized interview employs 
a specific list of questions to be asked sequentially. Tlie general interview uses an 
outline of issues, but die order of presentation may vary and the actual wording is 
not standardized. It is free-flowing and is regarded as “a speech event” in the 
words of Misliler (1986, p. 35). That was die style chosen for die study referred to 
in diis work. For example, die American participants were shown a chart with the 
headings Postives and Negatives; and the terms “indirectness,” (ground- 

work),” and 'Hatemae (stated position).” They were then asked, ‘"What are your 
positive experiences widi Japanese indirectness?” or ‘"Wliat are die good points 
related to die indirectness of your counterparts?” or a similarly worded question. 



Differentiating Quantitative and Qualitative Research 

I will now compare in-depth interviewing specifically with quantitative re- 
search in general. 

Apart from a different emphasis on numbers and words, the greatest contrast 
between quantitative and qualitative investigation is die amount of control the 
investigator has over the research. In qualitative interviews, interviewees fre- 
quently take die topic and develop it, providing new direction and allowing the 
interview to change course and die interviewer to explore unexpected topics. 

Quantitative mediods begin widi a hypothesis and use a deductive process, 
whereas intei*views gadier data and inductively process it to an explanation. 
Instead of validating an explanation, interviews develop an explanation, often 
uncovering unexpected results. In die sample study, for instance, the Japanese 
participants’ opinions on American assertiveness were not what the interviewer 
had expected when the investigation was begun. 
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Anotlier function of tlie interview is to reveal a relationship between the 
differing views of tl^e interviewee and tlie interviewer (Kratliwohl, 1993). As the 
interviewer in tlie sample study, I felt the speech strategy of explicitness to be 
preferable to ambiguity because it fosters comprehension of tlie speaker’s stand. 
However, it would have defeated the purpose of the interview to argue with the 
Japanese interviewees who reacted negatively to explicitness: 

• “If they [Americans] give too much background information that’s bor- 
ing and insulting — ^when they confront you with an issue and use a lot 
of words to be very definite and you know all about it. But it’s a habit 
of Americans.” 

• “For you tlie verbal code is supreme; you want everything in writing, 
like your teaching contract. It’s almost as if you can’t taist anyone.” 

• “I find him patronizing and boorish but I guess you find him eloquent.” 

Unlike quantitative research, in qualitative research most of the creative work — 
coding and analyzing die revelations into useful slots and assigning meaning to 
them — follows the data collecdon. 

The objectives of positivistic research and interview research are diametri- 
cally opposed to each other. The former deals widi validation, while the latter — 
employing questions like, “Do you like or dislike die assertiveness of Ameri- 
cans? Can you describe your experience with that?” — has basically two goals: 
description and explanation. In die social sciences in particular, die goal of 
research is seen as exploring for “the development of explanatory concepts diat 
help you understand individual behavior and social processes, concepts that 
help make life intelligible” Qoncs, 1996, p. 7). To explain a preferred or ab- 
horred communication style is to give a reason for it — ^why people feel die way 
diey do. In the sample study, reasons were articulated by a middle-aged female 
Japanese instaictor who had lived and taught in Yarmoudi, Maine for six years 
as she explained why she had come to accept assertiveness as positive. 

Well, Tve never seen or met an American professor who is not assertive. I like it 
very much; I mean it’s very productive. I don’t have to guess to get their point. 
Tliey come out with a point; then I can be assertive myself, right? Assertive persons 
usually permit the other person to be assertive, too, and I enjoy the interaction. 

A communication scholar who studied in the US for several years and deals 
widi Americans often in his professional associations clarified: 

They assert their own opinion, I think, rather than evading the issue and I appre- 
ciate that fact. I take it positively. At a conference. I’d read a paper and one of the 
Americans came to me after the presentation and told me that he didn’t agree. 
You know, we had a very fruitful exchange of opinions and afterwards we be- 
came friends. That’s very difficult among Japanese. That was a positive experi- 
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ence. Aiid during the speech Americans challenged me. Sometimes that shows a 
reaction to what I’m talking about and I enjoy it. It shows someone’s listening to 
me, not sleeping. 



The Strengths of In-depth Interviewing 

Perhaps one of the most advantageous aspects of interviews is the fact tliat they 
are particularly successful in obtaining large amounts of data in relatively short 
periods of time. 

Open-ended interviewing is typically as close to a natural process as pos- 
sible. There is more freedom in the type of questions, in tlie wording of tlie 
questions, and in tlie choice of setting all of which adds to the naturalness. This 
in turn facilitates cooperation. 

In addition, the range of data is unlimited. Experiments begin with a hypotli- 
esis and seek to verify it whereas in-depth interviewers are not so restricted. 
This allows the researcher to explore and discover the topic jointly with the 
interviewees because of the unstrucaired way in which tlie participants express 
their own thoughts, witli a reduced likelihood of missing new and unreported 
details or concepts. In the sample study, for example, discoveries emerged from 
the data as I listened to Japanese opinions on American assertiveness: 

• “They have warmth because tliey come up to me and tell me what they 
think. I like tliat.” 

• “I see tliey go right up to some other Japanese teachers and state tlieir 
case. They exude confidence. It’s fine, I think. ...” 

• “It sure beats the antitliesis, being self-effacing, which is what my col- 
leagues [Japanese] try to do. That’s [the habit of] a lot of Japanese, and 
that’s not good.” 

These were not explanations 1 had thought of when I entered the inquiry. The 
wholehearted acceptance and preference of assertiveness was contrary to my 
expectations. 

Because the study focused on lengthy individual participation, I was able to 
better grasp the complexity and background of tlie individual’s experience. This 
was aided by face-to-face contact in which non-verbal behavior could be ob- 
served and noted. Moreover, the peculiar format of interviewing allowed imme- 
diate follow-up questioning for expansion, omissions, clarification, and 
contextualization. These latter features point to how interviewers affect the in- 
terview positively: even if unintentionally they give feedback via body language 
or prompts, encouraging more complete responses. 

Rich verbal descriptions which portray phenomena are characteristic as in 
these responses to the concept of nemawashi (groundwork): 
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Well, at least nemawashi saves the struggle. In Japan there’s no struggle; it’s smo- 
o-o-oth. If there were any arguing, it’s done privately. Back home they argue 
vociferously in public, at the meetings, and I used to like the challenge of the 
confrontation. There’s always two camps, the proponents and the opponents. We 
have a vote and the winners savor the victory, but the losers cause a disagreeable 
gap and grudgingly wait and plan viciously for the next vote. In Japan that dissen- 
sion is avoided. I’m turning Japanese. [He laughed.] 

I think nemawashi worked, for example, when my visa problems were sorted out. 

All six of the foreigners consulted each other on my behalf and then we got 
together and went en masse to the department head who in turn went to the 
personnel in charge of external affairs, who went to his superior, etc. It went all the 
way up and was then implemented route de suite. 

Readers of the published results also frequently identify with the quotations 

which have intuitive appeal for them. 



The Weaknesses of In-depth Interviewing 

The gravest weakness of interviewing is tlie amount of time it takes, not just for 
die interviewing itself, but also for setting up appointments, commuting to meet 
die participants, transcribing the tape recordings, and analyzing the data. 

Since a one hour recording can take a typist from two to four hours (depend- 
ing on the accent, grammar, articulation, rate of speech production, and so on) 
to type, all data in the sample study was transcribed by hired transcriptionists. 
Additionally, managing die mountains of material typed up presented huge 
problems. If a response comes at an average rate of 150 words a minute, a one- 
hour cassette can yield 9,000 words, or 36 pages of text (at 250 words per page). 
It can be an oveiwhelming task to sift dirough, dissect, code, and analyze. 

The presence of die recording mechanism can be a deterrent to free speech. 
In diis research example it was not, given the cooperativeness of the partici- 
pants and the frankness of their statements, for example: 

• “So you thought we are all dumb Japanese?” 

• “They illiterate [sic], on and on and on like a dainkard.” 

• “You maybe look and act like ronin (leaderless samurai). 

Another weakness is diat errors of commission can occur on the part of die 
interviewee and die interviewer. Tliere is no assurance that what is related by an 
interviewee is accurate. The interviewee may also be uncomfortable or fearful and 
dierefore unable or unwilling to speak frankly, and diere is no guarantee diat the 
interviewer will not (knowingly or unknowingly) influence die responses. 

Errors of omission can also occur. People are commonly unaware of — or 
unable to verbalize — circumstances in their lives, or may widihold incriminating 
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evidence. Failure on tlie part of die interviewer to question some participants in 
a way tiiat is meaningful to them may also yield parsimonious replies. The role 
of interviewer demands well developed listening skills and interpersonal com- 
munication skills which, if not present, can easily and seriously weaken the 
quality of data collected. 

Analyzing the data presents a whole set of pitfalls. Only two are cited here: 
(a) How do you explain unanticipated results?, and (b) Verbal data, even more 
tiian statistics, is open to various interpretations. 



Sampling of Interviewees 

It is always nice to find a way of doing things that will save you time and energy, 
provided that you do not sacrifice quality in die process. . . . Sampling is just such 
a time and energy saver. Furthermore, when done properly, it not only does not 
sacrifice quality, but may actually improve the quality of the data obtained (Jones, 

1996, p. 180). 

For in-deptii interviewing, die interviewer could choose a true representative 
sample (such as a random, stratified, or systematic sample) of the population of 
interest as if quantitative researcli were to be carried out, or vary that by engaging 
in non-probability sampling such as in die judgmental and purposive format. 

For die communication style investigation carried out in the sample study, 
die “purposive” sampling format was chosen. It is also called “judgmental,” 
“deliberate,” or “selective” because the researcher uses judgment in selecting 
individuals who will be instaimental in gathering data (Lonner & Berry, 1986, p. 
87). Patton also uses die term “purposeful,” and offers the rationalization diat by 
selecting cases for study in depth, “the logic and power” is revealed: one can 
learn a great deal about issues of central importance to the purpose of the 
study, dius the term “purposeful sampling.” He contrasted quantitative and quali- 
tative investigation on die basis of sample size and mediod: 

Perhaps nothing better captures the difference between quantitative and qualita- 
tive methods than die different logics that undeigird sampling approaches. Quali- 
tative inquiiy typically focuses in depth on relatively small samples, even single 
cases (n = 1), selected purposefully. Quantitative methods typically depend on 
larger samples selected randomly (1990, p. l69). 

For die sample study, the purposive method was chosen instead of random 
sampling to increase die depdi of data exposed and to illuminate the questions 
under study. In addition, “random or representative sampling is likely to sup- 
press more deviant cases ... as well as the likelihood that die full array of 
multiple realities [may be] uncovered” (Lincoln & Cuba, 1985, p. 40). An even 
greater danger of random sampling, however, is the opposite situation, namely, 
a profusion of extieme cases. 
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Purposive sampling involves choosing just those elements (each individual in 
die population is considered an element, or a unit of analysis) you want to inter- 
view widiout regard to generalizability. The selection of participants could be 
guided (as it was in die cited work) by die premise that die knowledge of self and 
die ability to communicate provides a rich base of data and has priority over die 
representativeness of die entire sample. That means diat the investigator could 
plan not to work with great numbers of deviants, diat is, the polar cases which 
would skew the results, provide less than useful material, or not be applicable to 
die population who would value die results of your study. In effect, purposive 
sampling can lead to a better representation of the intended consumer. 

Tlie strategy in the sample research was the following: 

1. I devised a sample frame (a tentative list) of all possible types of candidates 
from amongst North American native English speakers and native Japanese speak- 
ers who teach English at Japanese universities. The scope was delimited to “native- 
ness,” nationality, subject taught and occupation, institution, level of institution, 
and geographical location: native speakers from Japan and North America teach- 
ing English as professors at Japanese colleges and universities in Tokyo. 

2. From a basic sample, I reserved a list of aberrant entries such as young, 
new arrivals, potentially lacking in self-awareness who are often at die “Honey- 
moon Stage” of die culture shock curve (loving every novel aspea about Ja- 
pan); veiy old professors, particularly those about to retire; and those who 
claimed to be totally bilingual and bicultural. Interviewees had to meet certain 
other specifications. One general specification was diat Japanese subjects had 
to speak English well enough to discuss abstract topics in depdi. 

3. In addition, I wanted the sample to take into account the disproportionate 
number of males hired in Japanese universities. I wanted to make sure that 
females were included in appropriate numbers corresponding to their pres- 
ence. I treated each gender as a separate population, and took die same per- 
centage from each, thereby including more males than females. 

4. I sought a mix of participants who I knew well and those who were 
strangers. I felt that friends and acquaintances would provide context, facilitat- 
ing analysis; while those with whom I had not yet established a rapport might 
be less hesitant to provide stronger statements and more divergent positions 
(diis proved taie). 

5. After interviewing my core sample and formulating some generalizations, 
I tested the robustness of those generalizations via triangulation. I resurrected 
die aberrant list and selected subjects who might contradict my findings. 

Only a small portion of the number of potential candidates were chosen. 
The sample group consisted of 25 individuals of each racial group, totalling 50 
interviewees. 1 O 
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Administrative Techniques 

Both science and art are involved in die interview itself. The participant’s moti- 
vation, the manner of die researcher, and die setdng are amongst the most 
crucial features affecting the quality and validity of the data received, and diere 
is much that the interviewer can do to improve them. 

The participant can be motivated by skillful prompdng. Utterances such as 
“And?” and “Then?” and “Mm” can aid die process. Non-verbal communication 
can be employed like nodding, gestures of encouragement, and moving in 
your seat as if to get a better (ear) vantage point. Exploring laughter and 
tolerating silence are among the many good hints provided by Seidman ( 1991 ). 
Douglas stated that: 

In general, the more of a listener die interviewer is, the more sensitive, the more 
openly intimate, the more sincerely interested in and warmly caring about the other 
person, the more he can temporarily surrender to the experience and soul of the 
other person, the more successRil he will be as a creative interviewer (1985, p. 57). 

It seems obvious that die interviewer should be a good listener, but too often 
die researcher fails to take in the interviewee’s points. This becomes clear when 
the tapes are played back and missed opportunides to clarify stances or to prod 
the speaker are noticed. Interviewers should be masters at forgetting their frames 
of reference, attending to the interviewees, coming to die meedng prepared to 
be surprised and to learn sonietliing entirely new, and entering die interview 
anxious to have dieir preconceived nodons overturned. Tlie axiom of Heraclids 
applies, “If you expect not die unexpected, you shall not find tnith.” One of die 
most important atdtudes for the interviewer to adopt is diat the interviewee’s 
contributions are valuable and useful. 

Even with the guide type of interview, hypodietical examples have to be 
prepared to elucidate concepts which may not be understood. In die sample 
study, for example, examples of tatemae (stated posidon) were provided when 
necessary, in the cases where informants were unfamiliar with the term, or 
when contrasts were made. 

As the interviewer will not know many of the interviewees, it is probably 
best to let the participants choose die setting, with die prompt diat “situational 
encapsuladon” (Douglas, 1985 , p. 99 ) is important to avoid intemipdons and to 
foster reflection. Douglas also counseled diat, “die recorder is bodi a reassur- 
ance of the seriousness of your pursuit and a brutal teclinological reminder of 
the human separateness diat undermines die intimate communion you are try- 
ing to create” ( 1985 , p. 83 ). In die sample study, a recorder was used in each 
interview and was found to be indispensable. More speaker autonomy can be 
attained if the interviewees are in charge of the pause button allowing them to 
choose which statements are recorded. 
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There are certain research variables which must be controlled in botli quan- 
titative and qualitative investigation. Brown’s quantitative work (1988, pp. 33- 
35) mentions a few. The halo effect is “tlie tendency among human beings to 
respond positively to a person they like”(Brown, 1988, p. 33). Subject expect- 
ancy and researcher expectancy figure in botli. The reactivity effect is exempli- 
fied by the situation in wliicli tlie “subjects actually form or solidify attitudes tliat 
they did not have before [die research]” (Brown, 1988, p. 35). 

Odier useful guidelines are offered by Frey and Oishi (1995) on logistics (avail- 
ability of resources sucli as facilities) and data quality (e.g., confidentiality). 



Conclusion 

The underlying cultural or social motivations for a behavior, communicative or 
odierwise, are central and complex, and that is what makes investigation of 
human behavior bodi difficult and exciting. Which method to pursue is die 
challenge. There is no perfect method. 

Qualitative methodology — in particular interviews — have been overlooked 
for too long. The compelling strengths of inductive, intensive, individual, in- 
sightful interviews will propel diem to greater popularity and a higher status in 
education and odier social science fields in die twenty-first century as diey 
make valuable contributions in uncovering “the world of secrets” out there. 
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An Annotated Bibliography 



Dale T. Griffee 

Seigakuin University 



It is typical of an annotated bibliography to review a ratlier large number of books 
in an area and give one or two sentences describing each book, Tliis article will 
review a rather small number of books because fewer books on research metliod- 
ology are published tlian books in a content area but will review each book in 
more detail tlian is usual for most annotated bibliograpliies. 

The purpose of each review is to give the reader a brief introduction to the 
book. Each review, therefore, can be considered only a snapshot of the book. 
While it is not possible to do justice to the complexity of any of these books 
anymore than it is possible to adequately evaluate a human being on tlie basis 
of a snapshot, it is hoped that an idea of tlie scope of each book plus some 
sense of tlie uniqueness of each book will assist the reader in making an in- 
formed choice when considering actual purchase. Toward tliat end, each re- 
view will strive toward description rather than prescription. The books are re- 
viewed alphabetically by autlior. 

A total of nine texts are reviewed. A book icon indicates the beginning of a 
review. The book title is given followed by the author or authors followed by 
the publisher and the year of publication then by the number of pages. The 
stmcture of each review is to list the citation of each book, then divide die 
review into four parts: the primary audience and the purpose of the book, die 
organization of the book, extra goodies, namely, features that are unusual or 
which might be overlooked, and die bottom line or a short summary. 
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Tliis annotated bibliography consists entirely of books describing research metli- 
odology for teaching English as a second or foreign language. All die books are 
relatively easy to order and to purchase in Japan, All die books are TESL oriented, 
and no attempt was made to review books on research mediodology from die 
mainstream content educational field of English speaking countries. 

All of these books are helpful. For example they can help you understand 
research articles in die TESOL Quarterly and the JALT Journal A classroom 
teacher who decides to engage in classroom research needs the experience of 
engaging in the research process, but in addition also needs the assistance of a 
guide or consultant. Without the experience of doing hands-on research, the 
new researcher has no grounding and the explanations in these books tend to 
sound abstract. Without the help of a guide, however, one tends to get lost. To 
put it another way, none of diese books is a substitute for die experience of 
working with an experienced researcher. One of the most helpful experiences 
you can have is taking a course with the author of one of diese books at a 
university or a TESOL summer institute. Furthermore, none of diese books reads 
like a novel (i.e, most are not meant to be read from cover to cover). It is a good 
idea to approach these books with an actual problem or at least a quesdon. It is 
also a good idea to read die relevant sections in more than one book as differ- 
ent audiors explain dieir soludons in different ways. 

Finally, keep in mind that there is no perfect book on research mediodology. 
Each book reviewed here is a reflection of the author’s context, purpose, and 
understanding of what research is and what new researchers need to know. All 
of these books are helpful either in whole or in part and there is no book 
reviewed here that would not be helpful to have in a teacher’s personal library. 




Focus on the Language Classroom: An Introduction to Classroom 
Research for Language Teachers, DAUwright and K. Bailey. Cambridge: 
Cambridge University Press, 1991. 250 pp. 



Primary audience. This book was written in non- technical language for lan- 
guage teachers with little or no research background. The purpose of the book 
is to introduce classroom research to inexperienced teachers as well as experi- 
enced teachers who are not yet researchers so diey can use research techniques 
to begin to think like researchers in die exploration of their own teaching. This 
is not, however, a “how-to book.” Although including both, it stresses quandta- 
dve research over qualitative research, and provides a radier thorough review 
of research issues. 

Organization, The book is divided into five n)ain secdons widi two chapters 
for each section. Part one discusses the history of classroom research, part two 
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discusses what to take into account when starting research, part tliree discusses 
oral error, part four is about classroom discourse, and part five discusses student 
openness and what promotes or inhibits students’ openness to learning. 

Extra goodies. One of die surprises of this book, to me at least, is that it has 
two chapters devoted to error analysis. The authors seem to sense, probably 
rightly, that classroom language teachers are very interested in oral speech 
errors and how to research diem. Tlie background and the issues involved are 
thoughtfully discussed. The odier goodie is the suggestions for further readings 
which follow each chapter. Each suggestion is a gem and reading them is like 
having an experienced teacher give you personal, cogent advice on what to 
read and where to look for it. Tliese suggestions should not be overlooked by 
teachers new to classroom research. 

The bottom line. This book is hefty, but readable. It is especially helpful for 
teachers in the process of deciding to take a research perspective in dieir 
classroom. 



Understanding Research in Second Language Learning, J.D. 
Brown. Cambridge: Cambridge University Press, 1988. 219 pp. 

The primary audience for diis text is classroom practitioners, especially those 
who are terrified by numbers, cliarts, tables, and statistics. The aim of the book 
is to explain basic statistical terms; how charts, tables, and graphs work; and the 
appropriate use of research design. It is a book for the consumer radier than the 
producer of statistical studies. 

Organization, There are 13 chapters wliich I, not die audior, divide into four 
sections. The first section, chapter one, sets die tone for the book which is that 
while statistical studies are not die only kind of research or even the best kind; 
diey are one type and they are useful. The second section, chapters two to four 
discusses variables and scales (nominal, etc.) wliich are used to talk about how 
research is basically designed. Knowledge of diese scales will be useful later in 
the book because they are key to deciding what kind of statistics to use. The 
third section, chapters five to eight, gives a short background course in statis- 
tics. If you have had a good testing course, you can either skip diese chapters or 
use them for review. The fourdi section, chapters nine to diirteen, are the heart 
of the book. 

Chapter nine, “Statistical Logic,” explains the logic of statistical design. The 
next tliree chapters discuss the diree major types of statistics: correlation, com- 
paring means, and comparing frequencies. Each cliapter has a section on choosing 
die correct statistic and a chart which uses variables and scales as a way of 
locating the statistic you might want to use. For example, if your dependent 
variable is a single interval scale and your independent scale is also a single 
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intei-val scale, where those intersect on the chart you find “simple regression” 
and “Pearson r,” wliich are two statistical procedures you can use. All these 
terms and procedures are explained in tlie chapters in some detail. 

Extra goodies. The special goodie for the new researcher is tucked away in 
chapter five which has the title “Critiquing Statistical Studies” because that is, in 
fact, the content of tlie chapter. What that title does not tell you is that there is 
a complete layout and description of the research article format. Research ar- 
ticles have a set format wliich cannot lightly be ignored. As a researcher you are 
expected to know the differences between such categories as materials, proce- 
dures, analyses, and results. You are also expected to know where an appendix 
is placed. Clearly laid out with each category explained and illustrated, this 
chapter alone is worth the price of tlie book. Check tlie format against tlie 
Publication Manual of tlie American Psychological Association, listed below, 
because there are slight differences. 

the bottom line. A relatively easy to understand book for an introduction to 
statistics and tlie logic behind statistical saidies. This is a book to refer to when you 
are at the research design level trying to decide wliich statistical procedure to use. 



Research in the Language Classroom. C. Brumfit and R. Mitchell. 
London: Modern English Publications in association with The Brit- 
ish Council, 1991. 172 pp. 

Primary audience. This is not a textbook, in fact, it is not even a book. It is a 
collection of articles and tlie target audience is never explicitly stated. The pur- 
pose of the book, which is explicitly stated, is to survey major issues in class- 
room research. The influence of the British Council guarantees that this collec- 
tion represents a British point of view. What saves tliis collection from being 
hopelessly parocliial is that the contributors are widely distributed — Canada, 
Australia, Japan, U. K., and the U. S.; cover a wide spectrum of positions; and 
are highly qualified. 

Organization. There are 12 articles divided into three major sections, refer- 
ences follow each article and there is no index. To give you some idea of tlie 
autliors and articles, I will list the first five titles witli tlieir authors. These five 
articles constitute tlie first of tlie tliree major sections. General Issues: “The 
Language Classroom as a Focus for Research” by Brumfit and Mitchell; “The 
Teacher as Researcher” by David Nunan; “Ethnography: Bandaid, Bandwagon, 
or Contraband?” by Leo Van Lier; “Researching Classroom Language Learning” 
by Rod Ellis; and “Tlie I-language Approach and Classroom Observation” by 
Vivian Cook. These five articles alone would justify tlie price of the collection. 

Extra goodies. This is a rich collection, and I will comment on only four of 
the articles. The aiticle by Baimfit and Mitchell is as fine an introduction to tlie 
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current issues in research as I have read. “Investigating Learners’ Language” by 
Green and Hecht discusses in a very practical way some of the problems re- 
searchers will probably encounter. “Classrooms and Display Questions” by 
Banbrook and Skehan discusses theory-dien-research (top-down) and research- 
then-theoiy (bottom-up) perspectives, and concludes with some suggestions on 
how die two could be combined. “Second Language Classroom Research and 
Teacher Education” by Michael Long continues a discussion introduced by 
Banbrook and Skehan by outlining problems of second language classroom 
research and, by implication, what we should be doing. 

TJoe bottom line. This is a valuable collection of articles because taken as a 
whole it deals with issues which are on the research agenda and which novice 
researches need to be aware of. Those issues are the teacher as researcher, 
types of research and dieir applicability, and a rationale for both quantitative 
and qualitative research design. How these issues are put and the answers diat 
are given will form the new “common sense” of research and the research 
environment in which we will all have to live. 



The Research Manual, Design and Statistics or Applied linguistics. 
E. Hatch and A. Lazaraton. New York: Newbury House, 1991 - 628 pp. 

Primaiy audience. The authors state that their text is for novice researchers, but 
that is like saying diat the complete Oxford English Dictionary is for beginning 
readers. This book on statistics, research, and design is impressive. The purpose 
of the text is twofold: on the one hand, the authors want to promote statistical 
literacy as well as statistical production, and on tlie other hand, they want to 
promote research planning. 

Organization. The book has 14 chapters and five appendices divided into 
five sections. Section one is on planning your research project, section two is 
on describing data, section three is on comparing groups, section four is on 
describing relationships, and section five (tlie conclusion) consists of five ap- 
pendices which include statistical tables and lists of tlie formulas used in the 
book. Each of the 16 chapters has workbook features at the end of the chapter 
which includes activities and practice sections. If you are really a novice re- 
searcher it is doubtful you can understand tliese exercises without the help of 
a more experienced guide. On the other hand, it is not necessary to know the 
meaning of every word in the dictionary in order to read, even to read rather 
complicated material. 

Extra goodies. In a book such as this, die category of extra goodies does not 
exist because die book is so vast and complex diat most everydiing is not only 
there, it is systematically arranged. If, however, we change the topic of extra 
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goodies to Tm-glad-to-see-it, dien I would nominate chapter 18 tided, "Assump- 
tions of Statistical Tests.” This chapter begins widi a helpful discussion of reli- 
ability and validity, and concludes widi assumptions of specific statisdcal tests 
as well as sections on solutions when assumptions cannot be met. 

The bottom line. This is die mother of all ESL statistical research textbooks 
and if you are interested in a comprehensive discussion of the statistical proce- 
dures used in applied linguistics, diis text should be in your library. 



Approaches to Research in Second Language Learning. D, M. 
Johnson. White Plains, New York: Longman, 1992, 253 pp. 

Primary audiejice. The primary audience of this text is graduate students in 
SLA, FL, and bilingual education. This is a first book and no previous knowl- 
edge of research methods or statistics is necessary. This book was written to 
provide an overview of die major approaches to research in second language 
learning. The author seeks to provide a balance between the quantitative and 
qualitative approaches. 

Organization. Six approaches to research are discussed. Three approaches, 
correlational, experimental, and survey research are more quantitative while 
die other diree, case studies, ethnographic, and multisite and multimethod 
studies are more qualitative. Two of the approaches will be discussed here. 

Chapter 3, on "The Correlational Approach,” is 22 pages long and contains 
an introduction, 9 criteria for analyzing correlational studies, and two sample 
studies both analyzed using the 9 criteria. Included in die introduction is a 
definition and example of a correlational coefficient and a discussion of valid- 
ity and reliability in measurement, but no formulas or statistics are given. Tliis 
is tme for all chapters in this book. The one page conclusion discusses the 
major advantages and disadvantages of correlational research. The chapter ends 
with 30 reference citations. 

Chapter 6, on “Ethnographic Research,” is 27 pages long and is organized in 
a fashion similar to the chapter on correlation. The introduction discusses types 
and uses of ethnographic research, a section titled "Doing Ethnographic Re- 
search” discusses the role of theory, field techniques, and analysis. Ten criteria 
for assessing ethnographic reports are given with a sample stvidy analyzed in 
detail using the criteria. The chapter ends widi 65 reference citations. 

Extra goodies. This book has many surprises. Each chapter has outstanding 
references. There is a chapter on the teacher-researcher movement and an 
appendix listing resources. The purpose of die chapter on teachers as researchers 
is to explore questions rather than propose answers. Questions are raised such 
as: Why do teachers see present research as irrelevant?. What is a teacher- 
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researcher?, and What are the goals of teacher-researchers? Do not skip the 
appendix, which is in three parts. First, it lists indexes and abstracts tlirough 
which you can locate articles by topic or author. Then it lists reviews of re- 
search literature on particular topics, and finally a list of 33 journals each of 
which is briefly described. 

The bottom line is tliat this is a good “getting started” book, especially if you 
are not clear on the various types of research. Because it covers the main types 
of research in EFL today, it might be a book to look at when you are at tlie stage 
of forming your research question. 



Understanding Language Classrooms: A Guide for Teacher-initi- 
ated Action. D. Nunan. London: Prentice Hall, 1989. 183 pp. 

Primary audience. The target audience is teachers who are new to research. 
The purpose of the text is (a) to introduce educational — as opposed to linguis- 
tic — issues to ESL audiences, (b) to introduce the ins and outs of looking at 
classrooms, and (c) to present classroom research as a teacher renewal tool. 
Nunan uses (b) to accomplish (a) and (c). 

Organization. The book is divided into seven chapters followed by three 
appendixes. After chapter one which gives a context, the remaining chapters 
group themselves into pairs. Chapters two and three are on who to look at 
(chapter two on teachers and chapter three on learners), chapters four and five 
are on metliods of investigation or ways of looking (data collection and class- 
room observation methods), and chapters six and seven are on professional 
development or why look at all. Chapters three and four are especially impor- 
tant, Chapter three presents a readable discussion of three of the most impor- 
tant issues for teachers: SLA, tasks, and learner strategies, while cliapter four 
discusses qualitative data collection methods such as diaries, field notes, inter- 
views, and protocol analysis. 

Extra goodies. Appendix C is an introduction to statistics. Statistics bother lan- 
guage teachers. If we had wanted to study mathematics, we would have become 
engineers. Tliis appendix explains basic ideas as well as some current statistical 
application such as t-tests, analysis of variance, chi-square, and factor analysis. If 
tliese are strange sounding words, this appendix provides an introduction. 

The bottom line. This is a basic book, but the seeds are all there. If the whole 
idea of research makes you ill, start with this book. If this book were taken 
seriously by a substantial number of language teachers, it would be a radical 
book with revolutionaiy consequences. 





147 



156 



Dale T, Griffee 




Research Methods in Language Learning. D. Nunan. Cambridge: 
Cambridge University Press, 1992. 249 pp. 



Primary audience. Tliis book is for practitioners in the field as well as graduate 
students. No previous knowledge of research is presupposed. The text was 
written to inuoduce research methods and tools, to prepare classroom teachers 
to understand and to critique research papers, and finally to help new re- 
searchers develop their own research strategies. 

Organization. The book can be divided into two parts. The first part, chap- 
ters one to four, discusses research methods (experimental, ethnographic, and 
case study). The second part, chapters five to eight, discusses research tech- 
niques (classroom observation, introspective metliods, elicitation techniques, 
and discourse analysis). 

Nunan covers a broad range of issues and what you notice depends largely 
on what you are looking for. I appreciated his discussion of reliability and 
validity in chapters tliree and four, his discussion of surveys and use of ques- 
tionnaires in chapter seven, and his discussion of discourse analysis, conversa- 
tion analysis, and interaction analysis in chapter eight. 

Chapter 10 concludes the text by discussing how to develop a research 
question and how to conduct a literature review — two problem areas for new 
researchers. 

Extra goodies. This book has at least two extra goodies. The first bonus is a 
glossary of terms and the second is a chapter on program evaluation. Nunan 
provides a glossary of some 60 key terms used in research such as action 
research, ethnography, reliability, stimulated recall, and variance. This glossary 
is helpful to new researchers because these terms can not be found in a stan- 
dard dictionary. 

The second bonus is a chapter on program evaluation. Teachers have a 
strong stake in program evaluation, but generally they are not well trained in 
program evaluation which makes chapter nine welcome. Nunan’s rationale for 
including this topic in a research textbook is that evaluation of an institution’s 
program is, by liis definition, research. Nunan discusses the difference be- 
tween evaluation and assessment. He also includes a sample design for an 
evaluation study as well as an example of a case study. 

The bottom line. This text summarizes information in lists and charts as well 
as giving clear examples and is especially helpful in its discussion of validity. 
This is a well balanced text which introduces major research areas and re- 
search techniques. 
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Publication Manual of the American Psychological Association 
(4th ed.). Washington: American Psychological Association, 1995. 
368 pp. 

Primary audience. Autliors who prepare manuscripts for graduate level classes or 
for publication. Tliere are many reasons to have this manual in your library: (a) 
Both JALT and TESOL follow APA wliicli means tliat all manuscripts submitted to 
them must follow the APA manual, (b) in addition, most major journals in our field 
follow APA or a modified version of APA, (c) tlie manual is very complete and not 
only gives detailed examples of how to reference almost anytliing you can tliink 
of, it also gives instaictions for issues such as use of abbreviations, quotations, 
numbers, footnotes as well as how to make figures and tables. 

Organization. Seven major sections covering everytliing you ever wanted to 
know about writing a paper. 

Extra goodies. Not applicable. 

The bottom line. Don’t fight it, just order it! You won’t be sorry. Order from 
your bookstore or write to: Order Department, American Psychological Associa- 
tion, RO. Box 2710, Hyattsville, MD 20784 U.S.A. 




Second Language Research Methods. H. Seliger and E. Shohamy. 
Oxford: Oxford University Press, 1989. 270 pp. 



Primary audience. According to the authors the book is intended for teacliers 
new to research, but my impression is that it is not a text for complete begin- 
ners. This would be a text for a class with an experienced teacher or for new 
researchers with access to more experienced researchers who would be willing 
to act as mentors. 

Organization. There are 10 chapters widi chapters one and two serving as 
the introduction, and chapter 10 serving as die conclusion. Chapters diree and 
four are a context to research, chapter five discusses the research plan, chapters 
six and seven discuss research design and diapters eight and nine discuss data 
collection and analysis. If I were to give this book a subtitle, it would be “formu- 
lating a research plan,” in that, the authors believe diat preparation is key to 
research. The text includes a thorough discussion of the differences between 
quantitative and qualitative research aldiough one gets the feeling diat in their 
hearts the authors prefer qualitative researdi. References are given after each 
section wliich sometimes causes certain articles to be referenced more than 
once. For example, Michael Long’s article titled “Inside die Black Box” is refer- 
enced three times, once on page 112, once on page 134, and again on page 199. 
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Extra goodies. This is a tightly organized and interwoven text so tliat litde 
tends to be overlooked. Nevertheless, I found tlie appendix reference for fur- 
tlier reading to be helpful. 

The bottom line. Reading tliis book is like hacking your way through a dense 
jungle and finding the treasure. While tliis book would not be in my small bag 
of 10 books to take to a desert island, it would be in my larger bag of 30 books. 
This is a standard text in the field by two very competent researchers, and it is 
unusual to read an aiticle on second language research that does not reference 
this book. Key works to describe this book would be solid, turgid, insightful, 
and required. It is helpful when you are considering a research perspective or 
looking for guidance for a specific problem. 
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Classroom Teachers and Classroom Research 

Classroom Teachers and Classroom Research is the second collection of articles in 
the JALT Applied Materials series. The articles in this volume are devoted to 
teacher-initiated research with a special focus on the classroom, which we 
recognize as the crucible of teacher concern and creative energy. While many 
articles are published in our field reflecting the research of teachers, this is the 
first volume published by JALT that is solely intended to define the role of 
teacher-initiated research in foreign language education, and help teachers 
become researchers in their own contexts. 

Written by practicing classroom teachers in Japan, Classroom Teachers and 
Classroom Research includes: 



• Language Teaching and Research David Nunan 

• Where Are We Now? Trends, Teachers, 

and Classroom Research Dale T. Griffee 

• First Things First: Writing the Research 

Proposal Martin White & Diane White 

• Literature Reviews: Obtaining 

Perspective Paul Riley 

• Methods for the Research Challenged .... Robert Homan & Chris Poel 

• Designing a Language Study .James Dean Brown 

• Action Research: Something for Everyone Oreg Hadley 

• Ethnography Research Linda Donan 

• Designing Surveys for Language Programs James Dean Brown 

• Bare Minimum Knowledge for Understanding 

Statistical Research Studies Hiroto Nagata 

• In-Depth Interviewing as Qualitive Investigation Marilyn Books 

• An Annotated Bibliography Dale T. Griffee 



Classroom Teachers and Classroom Research is edited by Dale T. Griffee and David 
Nunan, internationally recognized as an authority in classroom research and 
classroom research methods. 
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